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jt2.  smsomo  mm^v  Acmm - 

I  Personnel  &  Training  Research  &  Program 
I  Office  of  Naval  Research 
'  Washington.  D.C. _ 


This  study  involved  the  development  and  implementation  of  the  Production, 

!  Implementation  Evaluation  and  Revision  of  Instructional  Modules  (PIERIM) 
j  model  for  the  design  of  individualized  instruction.  PIERIM  is  designed 
I  as  an  interactive  model  with  activities  explicitly  prescribed  for  the 
I  instructor  and  the  educational  technologist.  PIERIM's  purpose  is  to 

!  provide  a  means  by  which  the  content  of  existing  teacher  training  programs 

I  can  be  transformed  into  a  format  (i.e.,  instructional  modules)  which 

i  is  compatible  with  an  individualized  teacher  training  program. 

i 

:  The  PIERIM  model  was  compared  with  two  other  system  models  for  the  design 
of  instruction  (Briggs,  1970;  &  Dick,  1969)  on  the  basis  of  assumptions 
upon  which  the  ilwdel  is  based,  personnel  required  to  implement  the  model. 
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Intended  level  of  app11cat1on»  level  of  analysis*  and  activities  required 
to  Implement  the  model. 

PIERIM  was  Implemented  for  a  portion  of  an  elementary  education  course: 
three  weeks  of  traditional  Instruction  was  converted  Into  25  Instructional 
modules.  The  modules  were  Implemented  In  a  conventional  classroom  environment 
(N”19},  revised  on  the  basis  of  learner  performance,  and  then  Implemented 
In  a  self-instruction  environment  (N»28).  The  pre-and  posttest  performartcds 
of  the  two  groups  were  approximately  equal. 

AnJ[nstruct1onal  Support  System  (ISS)  was  designed  to  support  the  Implex, 
mentation  of  the  Instructional  modules  In  the  self-instruction  environment. 
Pour  ISS  computer  programs,  written  In  FORTRAN  IV,  produced  the 
Instructional  modules,  tests,  and  summaries  of  learner  performance. 

It  was  also  demonstrated  that  the  programs  could  be  used  to  produce 
modules  for  a  similar  course,  educational  psychology. 

A  sln^gle  Index,  the  Revision  Indicator,  was  developed  which  ranked 
the  set  of  instructional  modules  on  their  relative  need  for  revision. 

Two  sets  of  Revision  Indicators,  derived  from  the  performance  data. 

Identified  tfie  same  three  Instructional  mCdules  as  most  needing  revision. 

Rank  order  correlation  between  the  two  sets  of  Revision  Indicators  was 

r  =  .83. 
s 

Faculty  members  from  Elementary  Education  and  Educational  Research 
ranked  the  Instructional,  modules  on  the  basis  of  the  relative  Importance 
of  a  teacher  candidate  being  able  to  demonstrate  the  behavior  described 
by  tht  modulti.  Rank  ordtr  cornlatlon  between  the  two  sets  of  rankings 
was  .71  for  the  total  let  of  modulei.  Faculty  members  indicated  that 
If  they  were  to  teach  the  course,  the  set  of  modulei  would  represent 
approximately  60  percent  of  the  evaluatior  unit. 

Based  on  experience  gained  through  Implementation  of  the  model, 
the  activities  were  evaluated  and  PIERIM  (3nd  edition)  represents  an 
operational  definition  of  the  job  descriptions  for  the  Instructor  and 
educational  technologist.  The  major  Identified  weakness  of  a  set  of 
Instructional  modules  produced  by  the  PIERIM  model  remains:  the  rele¬ 
vance  of  the  set  of  Instructional  modules,  when  compared  to  a  specific 
teacher  competency,  cannot  exceed  thi  relevance  of  the  subject  imtter 
from  which  the  modules  wbre  derived. 

The  estimated  cost  o*  deiigning  and  producing  28  sets  of  the 
Instructional  modulss  and  tests  was  $768.52,  with  design  and  production 
costs  representing  83  percent  and  17  percent  of  the  total,  respectively. 
Designing  and  testing  the  ISS  computer  programs  cost  an  additional 
$434.00.  Based  on  the  eight  classrotxn  hours  devoted  to  the  Implementation 
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of  the  modules  and  tests  in  the  self-instruction  environment,  the  develop¬ 
ment  cost  is  approximately  $150  per  hour  of  instruction.  In  order  to 
estimate  the  cost  of  implenienting  PIERIM  with  faculty  members  rather  than 
graduate  students,  the  personnel  costs  were  doubled  and  the  resulting 
estimate  was  $230  per  hour  of  instruction.  Utilizing  the  $230  estimate, 
PIERIM  could  be  implemented  and  the  contents  of  an  education  course 
(i.e.,  30  class  hours)  transformed  into  instructional  modules  and  tests 
for  use  in  a  self-instruction  environment  for  approximately  $6,900. 
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THE  DEVELOPMENT  AND  IMPLEMENTATION  OF  A  MODEL  FOR  THE  DESIGN  OF 
INDIVIDUALIZED  INSTRUCTION  AT  THE  UNIVERSITY  LEVEL 

ABSTRACT 

This  study  involved  the  development  and  Implementation  of  the 
Production,  Implementation  Evaluation  and  Revision  of  Instructional 
Modules  (PIERIM)  model  for  the  design  of  individualized  instruction. 

PIERIM  is  designed  as  an  interactive  model  with  activities  explicitly 
prescribed  for  the  instructor  and  the  educational  technologist.  PIERIM's 
purpose  is  to  provide  a  means  by  which  the  content  of  existing  teacher 
training  programs  can  be  transformed  into  a  format  (i.e.,  instructional 
modules)  which  is  compatible  with  an  individualized  teacher  training 
program. 

The  PIERIM  model  was  compared  with  two  other  system  models  for 
the  design  of  instruction  (Briggs,  1970;  &  Dick,  1969)  on  the  basis  of 
assumptions  upon  which  the  model  is  based,  personnel  required  to  imple¬ 
ment  the  model,  intended  level  of  application,  level  of  analysis,  and 
activities  required  to  iimplement  the  model. 

PIERIM  was  implemented  for  a  portion  of  an  elementary  education 
course:  three  weeks  of  traditional  instruction  was  converted  into  25 
instructional  modules.  The  modules  were  implemented  in  a  conventional 
classroom  environment  (N"19),  revised  on  the  basis  of  learner  performance. 
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and  then  implemented  in  a  self-instruction  environment  (N»28).  The  pre- 
and  posttest  performances  of  the  two  groups  were  approximately  equal 

An  Instructional  Support  System  (ISS)  was  designed  to  support  the 
implementation  of  the  instructional  modules  in  the  self-instruction 
environment.  Four  ISS  computer  programs,  vvritten  in  FORTRAN  IV,  produced 
the  instructional  modules,  t^sts,  and  summaries  of  learner  performance. 

It  was  also  demonstrated  that  the  programs  could  be  used  to  produce 
modules  for  a  similar  course,  educational  psychology. 

A  single  index,  the  Revision  Indicator,  was  developed  which  ranked 
the  set  of  instructional  modules  on  their  relative  need  for  revision. 

Two  sets  of  Revision  Indicators,  derived  from  the  performance  data, 
identified  the  same  three  instructional  modules  as  most  needing  revision. 
Rank  order  correlation  between  the  two  sets  of  Revision.  Indicators  was 
r^  »  .83. 

Faculty  members  from  Elementary  Education  and  Educational  Research 
ranked  the  Instructional  modules  on  the  basis  of  the  relative  importance 
of  a  teacher  candidate  being  able  to  demonstrate  the  behavior  described 
by  the  modules.  Rank  order  correlation  between  the  two  sets  of  rankings 
was  .71  for  the  total  set  of  modules.  Faculty  members  indicated  that 
if  they  were  to  teach  the  course,  the  set  of  modules  would  represint 
approximately  60  percent  of  the  evaluation  unit. 

Based  on  experience  gained  through  implementation  of  the  model, 
the  activities  were  evaluated  and  PIERIN  (2nd  edition}  represents  an 
operational  definition  of  the  Job  descriptions  for  the  instructor  and 
educational  technologist.  The  major  identified  weakness  of  a  set  of 
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instructional  modules  produced  by  the  PIERIM  model  refitains:  the  rele¬ 
vance  of  the  set  of  instructional  modules,  when  compared  to  a  specific 
teacher  competency,  cannot  exceed  the  relevance  of  the  subject  matter 
from  which  the  modules  were  derived. 

The  estimated  cost  of  designing  and  producing  28  sets  of  the 
instructional  modules  and  tests  was  $768.52,  with  design  ahd  production 
costs  representing  83  percent  and  17  percent  of  the  total,  respectively. 
Designing  and  testing  the  ISS  computer  programs  cost  an  additional 
$434.00.  Based  on  the  eight  classroom  hours  devoted  to  the  implementation 
of  the  modules  and  tests  in  the  self-instruction  environment,  the  develop¬ 
ment  cost  is  approximately  $150  per  hour  of  instruction.  In  order  to 
estimate  the  cost  of  implementing  PIERIM  with  faculty  members  rather  than 
graduate  students,  the  personnel  costs  were  doubled  and  the  resulting 
estimate  was  $230  per  hour  of  instruction.  Utilizing  the  $230  estimate, 
PIERIM  could  be  implemented  and  the  contents  of  an  education  course 
(i.e.,  30  class  hours)  transformed  into  instructional  moduRes  and  tests 
for  use  in  a  self-instruction  environment  for  approximately  $6,900. 
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CHAPTER  I 


INTRODUCTION 

Saettler  (1968)  traced  the  history  of  instructional  technology 
and  placed  1960  as  an  approximate  datfe  for  the  use  of  the  phrase  "total 
systems  approach"  to  describe  the  interaction  of  men,  machines,  and 
resources  within  the  context  of  an  organization  in  terms  of  specific 
tasks  and  outcomes.  Kyans  (1964)  discussed  the  importance  of  viewing 
education  as  a  system  and  presented  a  research  approach  to  system 
study  and  design  in  the  planning  of  higher  education.  Recent  develop¬ 
ments  in  teacher  education  demonstrated  the  necessity  for  viewing  the 
preservice  professional  component  of  a  program  as  a  subsystem  of  higher 
education. 

LaGrone  (1964),  in  a  proposal  presented  to  the  American  Associa¬ 
tion  of  Colleges  for  Teacher  Education  for  the  revision  of  the  preservice 
professional  component  of  a  program  of  teacher  education,  stated; 

The  current  instructional  materials  will  not  meet  the  demands 
of  the  proposed  professional  curriculum  ....  To  realize  the 
potential  of  the  proposed  content  outline  new  materials  that 
capitalize  on  the  concept  of  an  instructional  system  approach 
will  be  needed.  A  task  of  this  magnitude  would  be  difficult 
but  far  from  impossible  (p.  60). 

Silberman  and  Carter  (1965)  described  the  deficiencies  in  the  present 
education  system  and  concluded  that  "prospective  teachers  are  not 
taught  to  take  pride  in  using  and  evaluating  instructional  tools  that 
are  effective  in  producing  desired  student  behavior  (p.  73)." 
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Engbretson  (1969)  conducted  an  analysis  and  evaluation  of  plans 
for  comprehensive  elementaty  teacher  education  models  submitted  to  the 
U.  S.  Office  of  Education  (USOE),  and  provided  the  following  chrono¬ 
logy  of  events: 

August,  1967--A  meeting  at  which  the  possibility  of  funding  the 
elementary  teacher  education  models  was  discussed  was  held  in 
Washington,  D.  C. 

October,  1967— Request  for  proposals  was  issued  by  USOE'. 

January,  1968— Eighty  proposals  were  received  in  response  to  the 
request  from  USOE. 

February,  1968— USOE  convened  an  evaluation  panel  which  considered 
over  twenty  proposals  and  recommended  funding  of  twelve  models.  Nine 
of  the  twelve  models  were  funded  by  the  Bureau  of  Research  to  conduct 
feasibility  studies. 

Clarke  (1969)  reviewed  the  major  features  of  the  nine  funded 
proposals  and  made  the  following  comments  concerning  the  models  pro¬ 
posed  by: 

1.  Florida  State  University— "The  program  calls  for  staff  develop¬ 
ment  for  the  staff  of  the  teacher  education  institution  in  such  areas 
as  programmed  instruction  .  .  .  individually  prescribed  instruction, 
multi-media  and  simulation  techniques  (p.  287}." 

2.  University  of  Georgia--"Detai1ed  performance  characteristics 
are  specified,  but  corresponding  materials,  treatments,  etc.,  designed 
to  develop  each  characteristic  are  not  Included  (p.  288)." 

3.  Michigan  State  University— "The  program  anticipates  the  wide¬ 
spread  use  of  pupil  'instruction  material  packages'  (p.  290)." 


I 

I 

I 

I 

I 

I 

I 

! 


I 


3 


4.  University  of  P1ttsburgh“'"Tfi1s  Is  a  training  model  for  Indl- 
vlduellzed'instructlon.  The  program  wIlTuse  Individualized  Instruc¬ 
tion  to  prepare  teachers  In  order  to  facilitate  Individualization  of 
Instruction  within  any  school'  (p.  290)1" 

Sllberman  and  Kool  (1969)  described  a  generalized  model  based 
on  their  analysis  of  the  nine  funded  model  proposals.  Their  general¬ 
ized  model  Includes  sets  of  specifications  for:  (1)  curriculum  modules, 
(2)  student  flow  and  (3)  support  systems.  A  curriculum  module  Is 
defined  to  consist  of  a  single  objective.  The  time  required  to  com¬ 
plete  a  module  Is  considered  to  be  dependent  upon  the  task(s)  Involved 
and  the  strengths  and  weaknesses  of  each  Individual  trainee.  Within 
the  support  systems,  the  program  support  system  Is  used  to  design,  con¬ 
struct,  and  test  Instructional  modules,  and  to  handle  the  logistics  of 
the  program.  In  discussing  the  support  system,  Sllberman  and  Kool  (1969) 
stated: 

Only  a  few  of  the  models  consider  cost  factors  for  support 
systems.  Pittsburgh  estimated  that  It  would  cost  $804,000 
to  develop  materials,  and  would  require  20  hours  of  retrain¬ 
ing  per  staff  member.  They  also  estimated  that  they  would 
ne^  one  additional  staff  member  per  50  students,  and  50 
more  square  feet  per  pupil  (p.  7). 

The  specification  of  objectives  for  the  models  were  concluded  to  be: 

(1)  neither  specific  enough  nor  operational  enough  to  be  Implemented 
without  further  definition,  (2)  derived  from  or  compatible  with  the 
present  course  structure  In  the  better  teacher-training  Institutions, 
and  (3)  derived  from  content  analysis  rather  than  an  analysis  of  what 
It  takes  to  change  pupil  behavior  (Sllberman  &  Kool,  1969). 
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C.  Lange  (1968)  concluded  that  "In  order  to  introduce  changes 
in  teacher  education,  it  is  likely  that  programs  of  instruction  must 
be  developed  to  achieve  specific  sets  of  objectives  .  .  .  (p.  16)." 
Smith,  Cciien,  and  Pearl  (1969)  stated  the  problem  in  a  slightly  dif¬ 
ferent  manner: 

There  is  now  no  set  of  training  situations  available  to 
teacher  educators.  There  are  lists  of  objectives,  tests 
for  assessing  the  cognitive  achievement  and  attitudes  of 
trainees,  and  scales  of  rating  their  teaching  behavior. 

There  are  all  kinds  of  pretentious  models  for  teacher 
education.  But  there  are  no  materials  to  be  used  in 
actually  training  the  teacher  .  .  .  (p.  77). 

The  paradox  of  teacher  educators  talking  about  and  advocating 
individualized  instruction  and  yet  failing  to  individualize  the 
teacher  education  programs  was  noted  by  Carr  (1962)  in  the  early  1960 's, 
and  Yee  (1969)  continued  to  appeal  for  the  individualization  of  teacher 
education  programs.  Progress  toward  Individualization  of  teacher  edu¬ 
cation  programs  has  been  indicated  by  Monson's  (1969)  conclusion  that 
a  common  element  of  the  nine  elementary  teacher  models  is  "greater 
stress  on  individualization  .  .  .  (p.  101)."  Esbensen  (1968)  and  Baird, 
Belt,  and  Holder  (1968)  described  teacher  training  programs  which  have 
been  individualized  at  St.  SchoUstica  College  and  Brigham  Young  Uni¬ 
versity,  respectively. 

In  sumnary,  new  models  for  programs  of  teacher  education  con¬ 
tinue  to  be  developed  and  discussed  (LeBaron,  1969b)  and  orw  problem 
area,  if  not  the  major  problem  area  for  educators  concerned  with  the 
development  of  individualized  preservice  programs  of  teacher  education, 
appears  to  be  related  specifically  to  the  program  support  system 
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described  by  Silberman  and  Kooi  (1969).  The  functions  of  the  program 
support  system  include  the  design,  production,  and  testing  of  instruc¬ 
tional  modules  and  the  handling  of  the  program  logistics. 


Research  Methodology  and  Procedures 
There  is  a  growing  awareness  that  educational  researchers  must 
consider  research  methodology  and  procedures  which  differ  significantly 
from  traditional  methods.  Some  of  the  leading  educational  researchers 
are  reconsidering  the  entire  field  of  educational  research  and  the 
relationships  which  exist  among  the  developing  research  designs  and 
traditional  research  designs. 

Recent  statements  of  the  National  Acadeiiy  of  Education  (Cronbach 
&  Suppes,  1969)  provide  support  for  departing  from  the  traditional 
research  design: 

Too  many  writers  seem  to  limit  the  term  "research"  to  quan¬ 
titative  empirical  inquiry  .  .  .  The  study  of  education 
requires  non-quantitative  as  well  as  quantitative  techniques 
(p.  14). 

Decision-oriented  "product  research"  is  part  of  an  effort 
to  develop  an  educational  procedure  that  can  be  followed 
systematically  in  the  future,  ordinarily  in  many  localities. 

Such  product  research  is  often  called  "development"  (p.  27). 

Developmental  research  is  untidy*  It  is  disciplined,  in 
that  the  Investigator  is  expected  to  be  systematic,  so  that 
other  qualified  persons  can  follow  his  reasoning.  But  the 
process  is  one  of  reacting  rationally  to  the  unexpected. 

Though  the  innovator  may  be  sure  what  general  form  his  pro¬ 
duct  will  take,  he  will  soon  find  himself  deep  in  problems 
that  call  for  engineering  studies,  inventions,  or  funda¬ 
mental  scientific  inquiry  (p.  174-5). 

The  research  procedures  can  also  be  classified  under  the  original 

meaning  of  action  research  that  Travers  (1962)  attributes  to  Kurt 
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Lewin:  "research  in  a  real-life  situation  in  contrast  to  research 

within  a  laboratory  .  .  .  (p.  542)."  Travers  (1962)  concluded  that: 

There  is  a  need  to  develop  miniature  theoretical  systems 
related  to  events  in  the  classroom,  and  perhaps  this  type 
of  exploration  is  more  urgently  needed  than  attempts  to 
apply  current  learning  theories  to  classroom  situations 
.  .  .  (p.  555). 

Within  Scriven's  (1967)  categories  of  process  research,  this  study 
would  be  classified  as  formative  evaluation.  Formative  evaluation 
is  designed  to  discover  deficiencies  and  successes  in  intermediate 
versions  of  instructionol  materials. 

Stake  and  Oenny  (1969)  addressed  themselves  directly  to  the 

problems  of  research  as  it  applies  to  the  design  and  evaluation  of 

instructional  materials.  They  expressed  the  hope  that: 

.  .  .  individual  researchers  will  attempt  to  produce  needed 
analytical  techniques  to  assess  the  structure  and  coverage 
of  the  content  of  instructional  materials  as  well  as  to  pro¬ 
duce  performance  criteria  for  evaluating  the  behavior  of 
users  of  such  materials  under  a  variety  of  specifiable  con¬ 
ditions  of  use  (Stake  &  Denny,  1969,  p.  379). 

Statement  of  the  Problem 

The  present  study  consisted  of  three  separate  but  related  parts: 

1.  A  model  for  the  Production,  implementation.  Evaluation,  and 
^vision  of  instructional  Modules  (PIERIH)  was  designed  and  a  detailed 
non-empirical  comparison  between  PIERIM  and  two  existing  system  models 
for  the  design  of  Instruction  was  presented. 

2.  The  Implementation  of  PIERIM  for  a  portion  of  an  undergraduate 
elementary  education  course  was  conducted  in  order  to  determine:  (a) 
the  feasibility  of  implementing  the  model,  (b)  Job  descriptions  for 
the  participants  of  the  ia^lementation,  (c)  cost  estimates  in  terms  of 
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time  and  materials  required  to  implement  the  PIERIM  model,  (d)  evalua¬ 
tion  of  student  performance  when  utilizing  instructional  modules  pro¬ 
duced  using  the  PIERIM  model,  and  (e)  reconmendations  concerning  the 
contents  of  training  materials  that  could  be  used  to  develop  the  skills 
identified  for  the  participants  in  the  implementation. 

3.  Development  of  an  Instructional  Support  ^stem  (ISS)  which 
utilized  existing  technology  to  support  the  implementation  of  the 
PIERIM  model.  The  ISS  development  was  analyzed  to  determine:  (a)  ele¬ 
ments  required  in  an  ISS,  (b)  adequacy  of  the  ISS  as  measured  by  the 
output  of  the  system,  and  (c)  cost  estimates  in  terms  of  time  and 
materials  required  to  implement  the  ISS. 

Significance  of  the  Study 

The  significance  of  this  study  is  in  the  development  and  eval¬ 
uation  of  the  PIERIM  model  for  the  design  of  self-instructional  modules. 
The  PIERIM  model  provides  educators  with  an  alternative  system  model 
for  the  design  of  instruction  which  is  specifically  designed  to  mini¬ 
mize  the  number  and  complexity  of  nev/  skills  or  tasks  initially  required 
to  be  performed  by  the  participating  instructor.  The  PIERIM  model  is 
a  selection  model  as  contrasted  to  the  design  models  for  development 
of  Instructional  materials. 

The  study  did  not  attempt  to  compare  the  actual  development  of 
instruction  using  each  of  three  selected  models,  but  did  compare  the 
sImilaHtles  and  differences  in  procedures,  as  defined  by  each  model. 

The  detailed  comparison  of  the  models  provides  educators  with  some  of 
the  questions  which  should  be  asked  of  any  model  selected  for  the 
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design  of  instruction.  The  detailed  job  descriptions  that  were  prepared 
for  each  member  of  the  implementation  team  were  refined  and  the  PIERIM 
model  was  revised  on  the  basis  of  experience  gained  through  Implemen¬ 
tation  of  the  model . 

In  summary,  the  study:  (1)  adds  to  existing  bodies  of  knowledge 
(e.g.,  models  for  the  design  of  instruction  and  Instructional  support 
systems);  (2)  provides  an  alternative  model,  PIERIM,  for  the  design  of 
instruction;  (3)  provides  an  instructional  support  system  capable  of 
supporting  the  Implementation  of  the  PIERIM  model;  and  (4)  Is  topical 
in  relationship  to  USOE's  interest  and  support  of  the  design  and 
implementation  of  models  for  elementary  teacher  education  training  pro¬ 
grams. 

Of  particular  interest  to  educators  at  small  colleges,  junior 
colleges,  and  public  schools  Is  the  development  and  Implementation  of 
an  Instructional  Support  ^stem  (ISS)  which  utilized  existing  tech¬ 
nology  to  support  the  implementation  of  a  system  model  for  design  of 
instruction.  Each  part  of  the  study,  and  the  study  In  toto.  Is  of 
specific  interest  to  and  provided  new  knowledge  for  those  persons  and 
Institutions  which  are  Interested  In  the  use  of  system  models  for  the 
design  of  Instruction  and  the  desl^  of  Instructional  support  systems 
to  support  the  Implementation  of  the  wdels. 


CHAPTER  II 


SURVEY  OF  LITERATURE 

The  design  and  implementation  of  a  model  for  design  of  instruc¬ 
tion  requires  that  the  researcher  must  be  aware  of  the  current  research 
efforts  in  a  large  number  of  related  areas  of  research. 

The  current  efforts  in  the  field  of  curriculum  are  reviewed  for 
the  purpose  of  defining  a  curriculum.  The  implementation  of  the  PIERIM 
model  results  in  the  production  of  a  set  of  instructional  materials  for 
a  specific  content  area.  Having  defined  a  curriculum,  it  will  be  possi 
ble  to  determine  if  the  instructional  materials  produced  through  the 
implementation  of  the  PIERIM  model  constitutes  a  curriculum  for  the 
content  area.  The  existence  of  a  curriculum  and/or  a  set  of  instruc¬ 
tional  materials  implies  that  there  exists  an  instructional  system  in 
which  to  implement  the  materials.  By  defining  an  instructional  system 
and  reviewing  the  subsystems  which  comprise  the  instructional  system, 
the  PIERIM  trodel  and  the  Instructional  Support  System  can  be  related 
to  a  more  general  model  of  an  Instructional  System. 

The  1eai*ners  who  interact  with  an  instructional  system  can  have 
their  performance  evaluated  either  by  the  use  of  norm-referenced 
measures  or  criterion-referenced  measures.  Mager's  (1962)  definition 
of  behavioral  objectives  has  as  one  of  its  elements  that  a  standard  of 
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performance  must  be  included.  The  use  cf  behavioral  objectives  combined 
with  the  intended  use  of  the  instructional  materials  in  an  individual¬ 
ized  instruction  program  dictates  that  the  mastery  model  and  criterion- 
referenced  measures  are  both  specifically  related  to  the  PIERII<I  model. 

By  defining  formative  evaluation  and  discussing  its  application  to  the 
evaluation  of  learners,  behavioral  objectives,  test  items,  instruc¬ 
tional  matbri^ls  and  learning  environments,  it  is  possible  to  discuss 
the  relative  strengths  and  weaknesses  of  the  evaluation  procedures 
used  during  the  implementation  of  the  PIERIM  model. 

The  stated  purpose  of  the  PIERIM  model  is  to  design  instruc¬ 
tional  materials  which  can  be  used  to  support  an  individualized  teacher 
training  program.  The  research  related  to  individualized  instruction 
and  instructional  packages  is  directly  applicable  to  the  design  and 
implementation  of  the  PIERIM  model.  An  individualized  program  of 
instruction  requires  greater  use  of  the  educational  technology  that  is 
currently  available  to  educators.  Specifically,  applications  of  com¬ 
puters  and  the  use  of  audio  tapes  are  applicable  to  the  Instructional 
Support  System  which  is  designed  to  support  the  implementation  of  the 
PIERIM  model . 

Innovations  in  education  are  reviewed  in  order  to  identify 
potential  obstacles  to  the  implementation  of  the  PIERIM  model  in  a 
teacher  training  program  and  to  place  the  current  efforts  in  the  field 
of  system  models  for  design  of  instruction  into  a  chronological  per¬ 
spective. 
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Current  Interest  in  Curriculum 

Representative  of  the  current  interest  and  emphasis  on  the  entire 
field  of  curriculum  are:  (1)  a  discussion  of  the  problems  of  curriculum 
evaluation  by  Tyler,  Gagne^,  and  Scriven  (1967);  (2)  a  book  of  readings 
related  to  the  curriculum  of  the  public  schools  by  Short  and  Marconnit 
(1968);  (3)  an  issue  of  Review  of  Educational  Research  entitled 
Curricul um.  edited  by  Popham  and  Baker  (1969);  and  (4)  major  portions 
or  entire  issues  of  professional  journals  devoted  to  problems  of  cur¬ 
riculum  (Educational  Technology,  1970a,  1970b;  Phi  Delta  Kappan,  1970). 

Tyler  (1970)  pointed  out  the  wide  range  of  definitions  and  views 
associated  with  the  term  curriculum  that  are  currently  in  use  by  edu¬ 
cators.  From  the  current  definitions  of  curriculum,  Gagne's  (1967) 
operational  definition  of  curriculum  was  selected  for  use  in  connection 
with  the  study: 

A  curriculum  is  specified  when  (1)  the  terminal  objectives 
are  stated;  (2)  the  sequence  of  prerequisite  capabilities 
is  described;  and  (3)  the  initial  capabilities  assumed  to 
be  possessed  by  the  student  are  identified  (p.  23). 

Instructional  Systems 

The  definition  of  an  instructional  system  was  derived  by  com¬ 
bining  Corey's  (1967)  definition  of  instruction  and  Ryans'  (1964;  1965) 
definition  of  a  system.  An  instructional  system  is  defined  as  an  identi' 
fiable  assemblage  of  complexly  organized  elements  which  are: 

1.  Interrelated  by  process  and/or  structure. 

2.  Interdependent. 

3.  United  by  a  common  information  network. 
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4.  Characterized  by  a  regular  and  orderly  form  of  interaction. 

5.  Able  to  function  as  an  entity  to  enable  the  learner  to  emit 
or  engage  in  specified  behaviors  under  specified  conditions 
or  as  response  to  specified  situations. 

There  does  not  exist  in  the  literature  an  agreed  upon  set  of 
elements  or  subsystems  which  comprise  an  instructional  system.  The 
most  common  subsystems  discussed  can  be  classified  within  the  following 
scheme : 

1.  Design/Selection  of  Instructional  Materials  Subsystem 
provides  for  the  acquisition  of  instructional  materials 
(Bratten*  1969;  Deterline,  1968;  Flanagan,  1969;  Johnson, 

1967;  Silber,  1970). 

2.  Implementation  Subsystem  provides  the  environment  and  pro¬ 
cedures  for  the  interaction  between  the  learner  and  instruc¬ 
tional  materials  to  occur  (Deterline,  1968;  Johnson,  1967; 

P.  Lange,  1968;  Ryans,  1964;  Silvern,  1968;  Smith,  1966). 

3.  Evaluation  Subsystem  provides  the  means  by  which  changes  in 
other  subsystems  can  be  measured  (Flanagan,  1969;  Johnson, 

1967;  Silberman  &  Filep,  1968;  Tyler,  1951). 

4.  Revision  Subsystem  provides  the  decision  structure  for 
identifying  elements  or  subsystems  which  are  not  perform¬ 
ing  at  a  predetermined  level  of  performance  (Silberman  & 

Filep,  1968:  Smith,  1966). 

5.  Information  Management  Subsystem  provides  records  concern¬ 
ing  the  performance  history  of  other  specified  subsystems 
(Bratten,  1969;  Cooley  &  Glaser,  1968;  Flanagan,  1969; 

Smith,  1966). 

The  five  subsystems  are  considered  to  define  a  larger  instructional 
support  system  (ISS).  The  relationship  among  the  preservice  teacher 
education  system,  instructional  support  system  and  the  instruction 
system  are  shown  in  Figure  1. 
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PRESERVICE  TEACHER  EDUCATION  SYSTEM 


Instructional  Support  System 


Fig.  1.— Educational  systems  under  consideration 
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System  Models  for  Design  of  Instruction 
A  system  model  for  design  of  instruction  must  include  at  least 
the  following  procedures  in  the  order  indicated: 

1.  State  behavioral  objective. 

2.  Write  test  items  to  measure  the  behavior  stated  in  the 
behavioral  objective. 

3.  Select  or  design  instruction  to  achieve  the  behavioral 
object! ves . 

4.  Inclement  instruction  with  learners  from  the  intended  pop¬ 
ulation. 

5.  Evaluate  learner  performance. 

6.  Revise  instruction  and/or  procedures  based  on  the  learners' 
performance. 

The  current  state  of  the  art*  with  respect  to  system  models  for 
the  design  of  Instruction*  has  drawn  heavily  from  the  early  work  in 
programned  instruction*  military  training,  and  the  system  concepts 
developed  by  the  military  for  the  design  and  Implementation  of  new 
weapon  systems.  Morgan  and  Morgan  (1966)  considered  the  development 
of  programmed  instruction  as  an  educational  effort  that  was  analogous 
to  the  application  of  the  systems  approach.  Dick  (1969)  considered 
that  terms  such  as  "systems  analysis*"  "a  systems  approach*"  and 
"educational  technology*"  when  used  to  refer  to  the  process  Involved 
in  the  preparation  of  instructional  materials*  are  not  intended  to 
describe  the  learning  process. 

Dick's  (1969)  model  for  the  systems  approach  has  evolved  as  a 
set  of  procedures  which  can  be  followed  in  the  preparation  of  instruc¬ 
tional  materials.  Briggs*  Campeau*  Gagne'^  and  Nay  (1966)  described  a 
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recommended  procedure  for  choosing  instructional  media  for  each  stated 
behavioral  objective.  Briggs  (1970)  incorporated  the  procedures  for 
selecting  media  and  procedures  for  sequencing  instruction  (Briggs,  1968} 
into  a  general  strategy  for  the  design  of  instruction.  A  comparison 
between  Dick's  (1969)  and  Briggs'  (1970)  models  for  design  of  instruc¬ 
tion  reveals  that  they  contain  essentially  the  same  processes  but  are 
represented  in  different  degrees  of  elaboration.  Similar  models  for 
design  of  instruction  have  been  discussed  by  Bunderson  and  Butts  (1969), 
Childs  (1968),  Corey  (1967),  Cyrs  and  Lowenthal  (1970),  Eraut  (1967), 
Glaser  (1965;  1966;  1968),  Haney,  Lange,  and  Barson  (1968),  Homme, 
Csanyi,  Gonzales,  and  Rechs  (1969),  Kibler,  Barker,  and  Miles  (1970), 
LeBaron  (1969a),  Lindvall  and  Bolvin  (1967),  Tosti  and  Ball  (1969) 
and  Tyler  (1949).  The  work  of  Quinn  (1967;  1968),  Rundquist  (1967), 
Smith  (1966)  and  Thomas  (1970)  is  representative  of  the  work  that 
has  been  done  by  or  for  the  military  training  commands.  Moore  (1970) 
and  Ullmer  (1867;  1969)  have  discussed  models  as  they  apply  to  instruc¬ 
tion  in  institutions  of  higher  education. 

Irolementation  of  System  Models 

System  models  for  design  of  instruction  have  been  applied  to 
an  ever  Increasing  variety  of  instructional  environments.  The  two  best 
known  projects  to  apply  a  system  model  are  the  Program  for  learning 
in  Accordance  with  Needs  (PLAN)  (Flanagan,  1967;  1968}  and  Individually 
Prescribed  Instruction  (IPl)  (Lindvall  &  Bolvin,  1967).  Morgan  and 
Morgan  (1968)  described  the  role  of  system  models  for  design  of  instruc¬ 
tion  within  the  context  of  the  ES  '70  project. 
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Barson  (1967)  and  Haney,  et  al.,  (1968)  described  the  appli¬ 
cation  of  system  models  in  four  universities  which  involved  eight 
courses  each  with  a  minimum  annual  enrollment  of  500  students. 

Thornton  and  Brown  (1968)  reviewed  current  multi -media  projects  and 
applications  of  system  models  in  higher  education.  Additional  pro¬ 
grams  developed  for  in-service  teacher  education  were  reported  by 
Oeterline  (1968)  and  Miller  (1967). 

Government  and  military  projects  which  have  involved  the  appli¬ 
cation  of  system  models  include:  U.S.  Naval  Acadeii\y  (Quinn,  1967, 
1968;  Tosti  &  Ball,  1969),  Job  Corps  projects  (Harmon,  1969),  and 
instructor  training  for  the  Royal  Air  Force  (Thomas,  1970). 

Application  of  system  models  within  the  areas  of  computer- 
assisted  instruction  and  computer-managed  instruction  is  evident  in 
the  work  of  Bunderson  and  Butts  (1969),  Hagerty  (1970),  Kooi  (1968) 
and  Kooi  and  Coulson  (1967). 

The  system  models  for  design  of  instruction  generally  provide 
adequate  descriptions  of  the  processes  Involved  in  the  model  but  have 
generally  failed  to  provide  information  concerning  the  implementation 
of  the  model  such  as:  (1)  number  of  persons  required  to  implement  the 
model,  (2)  relationship  between  number  of  revision  cycles  and  the 
effectiveness  of  the  instructional  materials  produced,  (3)  costs 
involved  in  implementing  the  model,  and  (4)  definition  of  the  roles 
and  responsibilities  for  each  meeter  of  the  Implementation  team.  The 
work  of  0.  Markle  (1967)  and  Short,  Gcear,  Haughey*  4  Tien  (1968)  have 
demonstrated  that  when  a  system  model  for  design  of  Instruction  was 
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applied  to  a  training  problem  that  the  process  not  only  produced 
Instruction  which  was  more  effective  than  traditional  methods  but  also 
resulted  In  reduced  training  time  requirements.  Reports  of  the  work 
completed  by  Hagerty  (1970),  0=  Markle  (1967)  and  Short,  et  a1.,  (1968) 
demonstrate  a  trend  toward  more  complete  reporting  of  the  Implementa¬ 
tion  of  system  models  In  terms  of  resources  required,  activities  per¬ 
formed,  and  evaluation  of  the  learner's  performance. 

The  review  of  existing  system  models  for  design  of  Instruction 
revealed  that  the  knowledge  gained  from  the  present  study  woulc  pro¬ 
vide  new  and  relevant  knowledge  concerning:  (1)  an  analytical  compari¬ 
son  between  two  existing  models  and  PIERIM,  and  (2)  Information  con¬ 
cerning  the  actual  Implementation  of  the  model  In  terms  of  personnel 
Involved,  Job  descriptions,  materials  produced  and  cost  estimates  for 
each  of  the  system  conqMnents. 

Mastery  Models  and  Criterion-Referenced  Iteasures 
The  mastery  model  described  by  Mayo  (i970)  was  based.  In  part, 
on  earlier  work  by  Carroll  (1963)  and  Bloom  (1968a).  The  mastery 
model  Is  characterized  by  describing  Its  five  component  strategies: 

1.  Inform  students  about  course  expectations,  even  lesson 
'  ekpeCtatlons  or;ijnU  expectations,  so  that  they  view 

learning  as  a  cooperative  rather  than  as  a  competitive 
enterprise. 

2.  Set  standards  of  mastery  In  advance;  use  prevailing 
standards  or  set  new  ones  and  assign  grades  In  terms 
of  perfbrmance  rather  than  relative  ranking. 

3.  (He  short  diagnostic  progress  tests  for  each  unit  of 
Instruction. 
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4  Prescribe  additional  learning  for  those  who  do  not  demon¬ 
strate  initial  mastery. 

5.  Atten^st  to  provide  additional  time  for  learning  for  those 
persons  who  seem  to  need  it  (Mayo»  1970,  p.  2). 

Glaser  (1963;  1967),  Glaser  and  Cox  (1968),  and  Popham  and 
Husek  (1969)  discussed  not  only  the  similarities  and  differences 
between  norm-referenced  measures  and  criterion -referenced  measures, 
but  also  the  application  of  criterion-referenced  measures  to  evalua¬ 
tion  of  instruction.  A  criterion-referenced  test  was  operationally 
defined  to  include  any  measure  which: 

1.  Assesses  learner  performance  In  relation  to  a  predetermined 
standard  of  performance. 

2.  Provides  Information  as  to  the  level  of  performance  attained 
by  each  learner  which  Is  Independent  of  reference  to  the 
performance  of  other  learners  (after  Glaser,  1963  and 
Glaser  and  Cox,  1968). 

Criterion-referenced  tests  are  an  Integral  part  of  Bloom's  (1968)  and 
Mayo's  (1970)  mastery  models. 

Ebel  (1962)  discussed  ten  principles  which  should  be  considered 
when  tests  of  educational  achievement  were  being  prepared  and  used. 

The  first  five  principles  were  considered  to  be  equally  applicable  to 
criterion-referenced  measures: 

1.  The  measureAnt  of. educational  achievement  is  essential 
to  effective  education. 

2.  An  educational  test  Is  no  left  or  less  than  a  device  for 
facilitating,  extendihg,  and  refining  a  teacher's 
observation  of  student  achievement. 

3.  Every  Important  outcome  of  e^icatlon  can  be  measured. 

4.  The  most  Important  educational  achievement  Is  commiRd 
of  useful  knowledge. 

5.  Written  tests  are  well  suited  to  measure  ti.  student's 
coamiand  of  useful  knowledge  (p.  20-22). 


Based  on  the  results  reported  by  Bloom  (1968a),  the  implementa¬ 
tion  of  the  mastery  model  was  determined  to  be  appropriate  for  use  at 
the  university  level;  and  significant  improvement,  as  measured  by 
learner  performance,  was  reported.  In  reference  to  higher  education, 
Mitzel  (1970)  predicted  that  by  the  year  2000  "the  major  changes  will 
be  primarily  characterized  by  individualization  of  instruction  leading 
to  sophisticated  systems  of  adaptive  education  (p.  439)."  Glaser 
(1965)  concluded  that  the  analysis  and  specification  of  behavioral 
objectives  would  have  the  greatest  single  impact  on  improving  our 
education  system.  The  statement  of  behavioral  objectives  is  the  cru* 
cial  first  step  for  both  the  mastery  model  and  system  models  for  design 
of  Instruction. 

Three  of  Garvin's  (1970)  general  principles  can  be  used  to 
determine  the  applicability  of  cr1ter1on>referenced  measures  to  pro¬ 
grams  of  study  at  the  university  level: 

1.  Unless  at  least  one  of  the  Instructional  objectives  of  a 
unit  envisions  a  task  that  must  subsequently  be  performed 
at  a  specified  level  of  competence  In  at  least  some  situa¬ 
tion  .  .  . 

2.  If  public  safety,  ecwiomlc  responsibility*  or  other  ethical 
considerations  (tend  that  certain  tasks  be  performed  only 
by  those  "qualified”  for  them  by  formal  Instruction  .  .  . 

3.  In  any  Instructional  sequence  where  the  content  Is  Inherently 
cumulative  and  the  rigor  progressively  greater,  CRH  should 

be  used  to  control  entry  to  sucMSSive  units  .  .  .  (p.  7-8). 

When  Garvin's  (1970)  principles  are  applied  to  programs  of  teacher 
education.  It  Is  clearly  evident  that  criterion-referenced  measures 
are  applicable.  The  relevance  of  present  and  future  programs  of 
teai^er  education  depends  upon  the  ability  of  educators  to  specify 
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both  the  tasks  and  applicable  situations  which  must  be  performed  at 
a  specified  level  of  competence.  The  entire  set  of  behavioral  objec 
lives,  each  defined  by  an  observable  behavior,  applicable  situation, 
and  standard  of  performance  would  operationally  define  a  program  of 
teacher  education. 


Evaluation 


Definition 


tterwin  (1969)  reviewed  the  historical  development  and  changing 
concept  of  evaluation  and  concluded  that  "concepts  of  evaluation  have 
developed  in  response  to  needs  for  evaluational  practices  .  .  . 

(p.  25)."  The  combination  of  ideas  from  Stake's  (1967)  discussion 
of  curriculum  evaluation,  Scriven's  (1967)  discussion  of  formative 
evaluation  and  Wittrock's  (1969)  discussion  of  evaluation  of  instruc¬ 
tion  resulted  in  cne  following  definition: 

Formative  evaluation  is  the  collection,  processing,  and  inter¬ 
pretation  of  data  for  the  purpose  of  describing  and  making 
Judgement  as  to  the  quality  and  appropriateness  of  behavioral 
objectives,  instructional  materials,  environments,  and  learner 
performance,  and  utHixing  the  results  to  make  decisions  con- 
cering  the  modification  of  the  instructional  system  from  which 
the  data  was  derived. 

Modification  of  a  system  based  on  data  derived  from  the  system 
(e.g.,  output)  Implies  feedback.  Feedback  has  generally  been  defined 
as  any  output  of  a  system  which  either  directly  or  indirectly  serves 
as  future  input  to  the  system.  Within  the  context  of  a  mastery  model 
or  system  model  for  design  of  instruction,  the  role  of  the  evaluator 
is  to  utilize  the  output  of  the  system  to  identify  possible  weaknesses 
within  the  system  which,  if  corrected,  would  increase  the  efficiency 
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of  the  total  system  and/or  proportion  of  learners  attaining  the 
specified  standard  of  performance. 

Feedback  to  the  Instructor  provides  the  Information  required 
to  make  decisions  concerning  the  modification  of  Instructional  materl 
a1s  and/or  procedures  (Bloom,  1968a,  1969;  Cronback,  1963;  Glaser, 
1965;  Tyler,  1949,  1951;  Wittrock,  1969).  Feedback  to  the  learner 
has  been  recommended  by  Bloom  (1968a  &  b,  1969),  Evans  (1968),  Glaser 
(1965),  Guilford  (1968),  and  Wittrock  (1969)  for  the  purpose  of  allow 
Ing  the  learner  to  assess  the  adequacy  of  his  performance.  Specific 
recommendations  which  would  assist  the  learner  In  correcting  defic¬ 
iencies  or  selecting  his  subsequent  learning  activity  should  be 
Included  In  the  feedback  to  the  learner  (Bloom,  1968a,  1969;  Cook, 
1951;  Evans,  1968).  Formative  evaluation  and  the  resultant  feedback 
to  the  learner  have  been  reported  to  have  assisted  the  learner  by 
pacing  the  learning  and  motivating  the  learner  to  expend  necessary 
effort  to  confute  the  task  (Bloom,  1968a).  Bloom  (1968a,  1969) 
Insisted  that  when  formative  evaluation  procedures  are  utilized  the 
grading  should  be  based  on  a  predetermined  standard  of  performance. 

There  are  few  specific  guidelines  concerning  the  data  to  be 
collected,  techniques  for  analyzing  the  data,  or  decision  strategies 
for  assigning  priorities  to  the  changes  which  must  be  mads  to  an 
Instructional  system.  Recoawendatlons  are  reviewed  for  each  of  the 
following  elements  of  an  Instructional  system:  learner,  behavioral 
objectives,  test  Items,  Instructional  materials,  and  environment. 


Learner 


The  only  assumption  which  must  be  made  concerning  the  learner 
is  that  he  is  capable  of  learning  (Gagne^,  1965b).  Learning  is  an 
unobservable,  internal  process  of  the  learner,  and  learning  must  be 
inferred  from  changes  in  a  learner's  performance  before  and  after 
interacting  with  an  instructional  system  (Gagne^  1965b;  Wittrock, 

1969).  Learning  may  be  defined  as  "a  change  in  human  disposition  or 
capability,  which  can  be  retained,  and  which  is  not  simply  ascribable 
to  the  process  of  growth  (Gagne^  1965b;  p.  5)." 

Tyler  (1949)  identified  two  primary  methods  of  evaluating  a 
learner's  performance  as  paper  and  pencil  type  tests  for  the  evaluation 
of  cognitive  skills  and  the  collection  of  actual  products  of  the 
learner.  After  the  learner's  performance  had  been  evaluated,  an  appro¬ 
priate  unit  had  to  be  selected  for  collecting,  processing  and  reporting 
the  data.  Flanagan  (1951)  discussed  the  relative  merits  of  each  of  the 
following  methods  of  summarizing  learner  perfomance:  raw  score,  rank, 
level  of  development,  growth  and  profile.  Criterion-referenced  measures 
dictate  that  at  the  very  minimum  the  learner's  performance  must  be 
reported  in  relationship  to  a  predetermined  standard  of  performance 
(Cook,  1951;  Oeterline,  1967;  Evans,  1968;  Lindvall  &  Cox,  1969; 
Wittrock,  1969)  and  each  component  should  be  evaluated  separately 
(Cook,  1951;  Flanagan,  1969). 

S.  Markle  (1967)  and  Wittrock  (1969)  expressed  concern  for  the 
lack  of  reported  data  used  to  describe  the  characteristics  of  the 
intended  population  and/or  the  learners  who  were  utilized  during  the 
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formative  evaluation  of  instructional  systems.  It  has  been  recommended 
that  alternate  forms  of  the  evaluation  instrument  be  prepared  and  that 
the  learner  be  allowed  to  take  a  criterion-referenced  test  as  many 
times  as  required  to  achieve  the  standard  of  performance  specified 
(Bloom,  1968a,  1969;  Mayo,  1970).  Eraut  (1967)  and  Silberman  and 
Carter  (1965)  recommended  that  where  behavioral  objectives  are  sequen¬ 
tially  dependent  that  mastery  must  be  attained  by  a  learner  before  he 
is  allowed  to  attempt  subsequent  behavioral  objectives. 

Green  (1967)  and  Moxley  (1970)  concluded  that  variability  in 
learner  performance  is  attributable  to  the  relative  inefficiency  of 
the  instructional  materials.  Selection  procedures  of  learners,  the 
lack  of  required  prerequisite  behaviors  by  learners,  and  problems 
related  to  learner  motivational  or  emotional  problems  were  identified 
as  other  possible  sources  of  variation  in  learner  performance  (Deterllne, 
1967). 

Behavioral  Objectives 

Mager's  (1962)  definition  of  behavioral  objectives  was  selected 
as  an  adequate  definition  of  the  term.  Wittrock  (1969)  suggested  that 
more  complete  descriptions  be  made  of  the  conditions  under  which  the 
learner's  behavior  would  be  evaluated. 

The  need  and  rationale  for  stating  educational  goals  In  terms 
of  learner  performance  has  been  thoroughly  discussed  by  Gagne^( 1965a  & 
1968a)  and  Tyler  (1949;  1951;  1964).  Research  studies  have  reported 
direct  benefits,  in  terms  of  learner  performance,  from  simply  stating 
the  desired  behavioral  objectives  and  then  emphasizing  the  attainment  of 
those  objectives  (Mager,  1968b;  Mager  S  McCann,  1961;  McNeil,  1966; 
Wittrock,  1962). 
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Two  major  problems  have  been  identified  in  relation  to  behaviLi'al 
objectives:  (1)  How  to  evaluate  the  behavioral  objectives,  and  (2)  A 
rationale  for  establishing  a  standard  of  performance.  Scriven  (1967) 
reconinended  that  external  judges  be  used  to  evaluate  the  relative 
importance  of  the  behavioral  objectives  and  the  congruence  between  the 
test  items  and  the  behavioral  objectives,  iloxley  (1970)  idenLified 
the  receiving  system  as  a  potential  judge  in  the  evaluati  r.,  o'  '.ohavi- 
oral  objectives.  In  terms  of  a  preservice  teacher  educatiot,  ^'riHji’uin, 
the  public  schools  v.'ould  be  used  as  a  judge  in  the  evaluation  of  tne 
behavioral  objectives  of  the  program. 

There  appears  to  be  three  alternatives  for  establishing  stari* 
dards  of  performance:  (1)  insist  upon  near  perfect  i.iastery  (..reeii,  1967), 
(2)  select  an  arbitrary  minimum  standard  (Deterline,  1967),  or  (3) 
develop  a  rationale  for  setting  standards  of  performance  (Davis,  1951). 

An  adequate  standard  of  performance  was  identified  as  one  -diicit  when 
achieved  by  a  learner  would  successfully  predict  that  lie  would  answer 
dll  remaining  items  correctly  (Davis,  1951). 

Test  Items 

System  models  for  design  of  instruction  and  mastery  niudols 
each  identify  the  first  concern  in  evaluating  test  items,  whicli  is  to 
establish  the  content  validity  of  the  item  (Bloom,  1968a;  Cronbach, 

1963;  Ebel,  1956;  Husek,  1969:  Popham  Husek,  1969;  Tyler,  1949; 
IJittrock,  1969).  When  test  items  are  derived  directly  f'rom  statements 
of  behavioral  objectives,  as  they  are  in  a  system  model  for  design  of 
instruction,  the  content  validity  of  the  item  has  been  establ  islied. 
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After  the  item  format  and  sample  test  situations  have  been 
agreed  upon,  the  next  step  is  to  start  the  construction  of  a  pool  of 
homogeneous  test  items  (Hively,  1970;  Lundin,  1970;  Tyler,  1967). 

Hively  (1970)  and  Lundin  (1970)  reported  procedures  that  they  have 
utilized  for  the  generation  of  pools  of  homogeneous  test  items  for 
the  field  of  mathematics. 

Empirical  testing  of  test  Items,  using  both  individual  and 
small  group  procedures,  has  been  recommended  by  Tyler  (1949).  The 
method  of  scoring  the  performance  of  a  learner  should  be  made  as 
objective  as  possible  (Bloom,  1969;  Lindvall  &  Cox,  1969;  Tyler,  1949; 
Wittrock,  1969)  and  the  basis  of  scoring  should  be  made  known  to  the 
learner  (Wittrock,  1969).  Evans  (1968)  recommended  the  use  of  multiple- 
choice  type  items  whenever  possible  and  contended  that  the  ultimate 
operational  definition  of  the  instructional  system's  objectives  is  the 
posttest  used  to  evaluate  the  learner's  performance. 

Cox  and  Vargas  (1966),  Glaser  and  Cox  (1968),  Hills  (1970), 

Husek  (1969),  Moxley  (1970),  Popham  (1970),  and  Popham  and  Husek  (1969) 
have  all  expressed  concern  because  of  the  lack  of  appropriate  methods 
of  analyzing  data  from  criterion-referenced  measures  of  learner  per¬ 
formance.  The  suggested  recoirmendations  have  been  very  general  in 
nature,  such  as:  the  proportion  of  learners  passing  an  item  should  be 
low  on  the  pretest  and  high  on  the  posttest  (Glaser  &  Cox,  1968;  Moxley, 
1970),  and  a  negative  discriminator  in  an  item  pool  should  be  carefully 
analyzed  (Popham  &  Husek,  1969).  Specific  procedures  for  item  analysis, 
based  on  the  pretest-posttest  design,  have  been  discussed  by  Cox  and 
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Vargas  (1966)  (e.g.,  pretest-posttest  difference  Index)  and  Popham 
(1970)  (e.g.,  fourfold  analysis  of  pretest-posttest  learning  states). 

Instructional  Material 

The  pretest-posttest  design  has  been  widely  recommended  and  is 
essential  If  learning  Is  to  be  Inferred  from  changes  in  the  learner's 
performance  before  and  after  Interacting  with  an  Instructional  system 
(Deterllne,  1967;  Glaser  &  Cox,  1968;  Lindvall  &  Cox,  1969;  Lumsdaine, 
1965;  Provus,  1969;  Tyler,  1949;  Wittrock,  1969).  The  pretest-posttest 
design  Is  considered  a  minimal  design  by  Tyler  (1949)  and  additional 
observations  of  the  learner's  performance  were  recommended  to  estimate 
the  retention  of  the  performance. 

When  the  only  data  available  to  an  evaluator  Is  from  a  pretest- 
posttest  design,  It  Is  exceedingly  difficult  to  determine  which  element 
of  the  Instructional  system  should  be  revised.  Provus  (1969)  described 
a  sequence  of  activities  which  were  used  in  evaluating  performance  data 
using  the  behavioral  objective  as  the  basic  unit  of  evaluation. 

The  results  of  any  evaluation  of  Instruction,  utilizing  the  one 
group  pretest-posttest  design  (Campbell  &  Stanley,  1963)  must  consider 
each  of  the  eight  classes  of  extraneous  variables,  which  affect  Internal 
validity,  as  potential  sources  of  change  In  learner  performance  which 
might  otherwise  be  attributed  to  the  Instructional  system.  Baker  (1969) 
reviewed  the  current  state  of  curriculum  evaluation  methodology  and  con¬ 
cluded  that  In  the  preceding  three  years  little  empirical  work  had  been 
reported.  The  field  of  curriculum  evaluation  remained  poorly  defined. 
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Environment 

The  environment  of  an  instructional  system  includes  the  pro¬ 
cedures  and  physical  facilities  required  to  accomplish  the  desired 
interface  between  the  learner  and  the  instructional  resources.  Bloom 
(1968a)  recognized  the  need  for  more  than  a  simple  description  of  the 
environment  when  reporting  the  evaluation  of  instruction.  The  need 
for  a  clear  statement  of  the  assumptions  made  concerning  the  instruc¬ 
tion  system  was  first  noted  by  Tyler  (1967).  Total  time  available  to 
the  learner,  resources,  sequence  of  instruction,  and  the  decisions 
and  emphasis  concerning  the  elements  of  the  instructional  system  were 
identified  as  representing  the  type  of  information  which  should  be 
Included  in  evaluation  reports  (Wittrock,  1969).  Dick  (1968)  suggested 
that  general  subjective  type  information  from  learners  concerning 
relatively  small  segments  of  instruction  would  be  useful  in  formative 
evaluation  of  instructional  s^ terns. 

In  the  past,  the  application  of  evaluation  techniques  and 
research  design  to  the  formative  evaluation  of  an  instructional  system 
have  been,  for  the  most  part,  theoretical  rather  than  practical.  Recent 
studies  by  Dick  (1968),  Hagerty  (1970),  0.  Harkle  (1967),  and  Short,  et 
al.,  (1968)  report  the  results  of  actual  developmental  and  evaluation 
studies  of  instructional  systmns.  The  recent  work  of  Stufflebeam  (1968, 
1969)  and  Wallace  and  Shavelson  (1970)  provide  much  more  detailed 
descriptions  of  the  logical  structures  of  evaluation  designs  which  are 
applicable  to  the  formative  evaluation  of  instructional  systems. 
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Ind1v1duaH2ed  Instruction 

There  does  not  exist  at  this  time  a  universally  accepted  defini¬ 
tion  of  the  term  "Individualized  Instruction."  Flanagan  (1967,  1968) 
presented  four  methods  by  which  Instruction  could  be  Individualized. 

These  Included  (1)  establishment  of  alternative  behavioral  objectives, 

(2)  placement  of  students  Into  the  program  based  on  their  entry  behavior, 

(3)  development  of  alternative  methods  of  Instruction,  and  (4)  pro¬ 
vision  for  each  student  to  progress  through  the  Instructional  program 
at  his  own  rate. 

Two  of  the  better  known  examples  of  Individualized  Instruction 
programs  are  the  Duluth  Program  (Esbensen,  1968)  and  the  Individually 
Prescribed  Instruction  (IPI)  program  at  the  Oakleaf  School  (Cooley  & 
Glaser,  1968).  The  IPI  program  determines  the  course  of  study  and  the 
goals  related  with  the  course  of  study.  The  student  Is  then  offered 
a  limited  choice  of  materials  and/or  methods  for  attaining  these  goals. 
The  Instruction  prescribed  for  a  student  In  the  IPI  program  Is  decided 
by  an  Instructor  on  the  basis  of  the  student's  demonstrated  achievement 
level.  Esbensen  (1968)  made  the  point  that  Individualized  Instruction 
and  Independent  study  are  not  synonomous  terms.  The  amount  of  Indepen¬ 
dent  study  Incorporated  In  an  Individualized  Instruction  program  depends 
upon  the  ability  level  of  the  students  Involved  and  the  requirements 
established  by  the  stated  behavioral  objectives.  Both  the  Individualized 
Instruction  programs  are  characterized  by  the  Inordinate  amount  of  paper 
work  required  to  manage  the  Instructional  system.  Both  the  Duluth  and 
the  IPI  programs  Initially  Involved  the  use  of  non-automated,  paper  and 


pencil  type  Information  management  systems  (Cooley  &  Glaser,  1968; 
Esbensen,  1968). 

Instructional  Packages 

The  transition  from  traditional  to  individualized  instruction 
has  been  seen  by  Diamond  (1968),  Loughary  (1968b)  and  Ubben  (1970)  as 
the  major  impetus  for  the  development  of  instructional  modules  or 
packages.  Diamond  (1968)  anticipates  that  within  every  discipline, 
series  of  self-contained  instructional  modules  will  represent  the  con¬ 
tinuum  of  instruction.  Instructional  packages  or  systems  have  been 
developed  as  supplementary  materials  but  there  appears  to  be  growing 
interest  in  the  development  of  complete  instructional  systems  for 
specific  areas  of  study  (Loughary,  1968b).  Ubben  (1970)  identified 
several  different  names  being  used  to  identify  Instructional  packages: 
contracts,  IPI,  Learning  Activity  Package  (LAP),  Teaching  Learning 
Unit  (TLU),  and  UNIPAC.  The  connon  elements  of  the  instructional 
packages  were  identified  by  Ubben  (1970)  as: 

Each  is  a  set  of  teaching-learning  materials. 

Each  package  focuses  on  a  major  leamable  idea,  skill  or 
attitude. 

Objectives  are  clearly  stated  in  behavioral  terms,  shifting 
the  emphasis  for  performance  from  the  Instructor  to  the  learner. 

Each  package  relies  heavily  on  the  use  of  learning  resources 
which  can  be  student-operated,  allowing  the  student  to  obtain 
Information  and  direction  without  the  constant  attention  of  the 
teacher. 

Each  package  attempts  to  use  a  variety  of  media  components  to 
provide  variation  in  instructional  modes. 

Each  package  has  student  evaluative  devices  including  pretests, 
student  selftests  and  posttest  (p.  31-32). 
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Educational  Technology 

The  national  conference  sponsored  by  Designing  Education  for 
the  Future:  An  Eight-State  Project  clearly  documented  the  scope  of  the 
problems  of  effectively  utilizing  technology  In  education  and  the 
potential  payoffs  If  It  can  be  accomplished  (Morphet  &  Jesser*  1968). 

In  the  past,  the  concept  of  educational  technology  has  been  related 
to  hardware  (I.e.,  slide  projectors,  television,  computers,  etc.). 

Recently  Gagne^  (1968b)  and  Bright  (1968)  among  others  have  proposed 
that  educational  technology  be  viewed  as  a  process  which  centers  on 
the  systematic  design  of  Instruction. 

Educational  technology,  as  It  relates  to  Instruction,  Is  defined 
for  this  study  as  the  processes  required  to  design  Instructional  modules 
(I.e.,  system  models  for  design  of  instruction)  and  the  hardware  required 
to  support  the  Implementation  of  the  Instructional  modules  (I.e.,  Instruc¬ 
tional  support  system).  Within  the  context  of  this  study,  a  system 
model  for  design  of  instruction  Is  used  to  produce  the  Instructional 
modules  and  the  computer  and  audio  tapes  are  two  principal  component  of 
the  Instructional  support  system  used  to  Implement  the  Instructional 
modules  In  a  self-instruction  ecviroranent.  Previous  uses  of  computers 
and  audio  tapes  In  Instructional  activities  are  reviewed  In  the  following 
sections. 

Sllberman  (1969)  and  Sllberman  and  Filep  (1968)  included  Instruc¬ 
tion,  research  and  development,  arr*  -  ^^.^^ement  as  three  operational 
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applications  of  computers  to  education.  There  are  many  computer-managed 
Instruction  programs  that  have  Incorporated  features  other  than  record 
keeping  as  part  of  the  management  systems.  Hany  of  the  projects  are 
still  In  the  developmental  stage  and  their  ultimate  configurations  have 
not  been  finalized.  Morgan  (1969)  has  classified  the  following  as  com¬ 
puter-managed  Instruction  (CMI)  projects: 

Harry  S1 1  barman's  work  with  the  Southv/est  Regional  Educational 
Lkboratory" and  the  Los  Angeles  Public  Schools;  Robert  Glaser 
of  the  University  of  Pittsburgh  working  with  the  Oakleaf 
School  in  Pennsylvania;  Donald  Torr  of  Sterling  Research  Insti¬ 
tute;  Don  Tosti  of  Westinghouse  Learning  Corporation  snd 
Alexander  Schure  of  New  York  Institute  of  Technology  all  of 
whom  are  working  with  the  Ul^.  Naval  Acadeiqy.  All  of  these 
projects  are  sponsored  by  tpe  U.S.  Office  of  Education.  Another 
large  project  Involving  CMI  Is  headed  by  John  Flanagan  under 
the  sponsorship  of  the  American  Institute  for  Research  and 
Westinghouse  Corporation  (p.  100.). 

Morgan  (1969)  concluded  that  the  simlllarltles  of  the  CMI  projects— 

(1)  the  design  of  learning  Interventions  based  on  carefully  stated 
behavioral  objectives  and  (2)  the  mediation  of  the  computer  between 
the  student*  his  performance  on  behavioral  objectives  and  the  Inven¬ 
tory  of  Instructional  resources  related  to  the  objecti ves— are  greater 
than  their  differences.  The  structure  of  a  D1!  system  written  In 
FORTRAN  programming  language,  which  has  been  Implemented  In  a  public 
school,  was  described  by  Steffenson  and  Read  (1970). 

The  potential  of  the  computer  as  an  Instructional  materials  data 
bank  or  computer-based  iMtructlon  was  discussed  by  Loughary  (19€8a). 
Blanchl  and  Burr  (1970),  Clayback  (1970),  Cross  (1970),  and  Elsele  (1970) 
report  the  operation  .  A  systems  and  research  conducted  utllizl''?  the 
Instructional  materials  produced  by  the  computer  systems.  The  mearch 
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studies  typically  report  no  significant  difference  In  learner  perform¬ 
ance  (Cross,  1970).  The  successful  application  of  computers  to  education 
depends  on  the  assignment  of  tasks  which  are  matched  to  the  characteris¬ 
tics  of  the  computer  which  the  computer  Is  uni  iuely  or  best  capable  of 
accomplishing  (Muller,  1968). 

Audio  Tapes 

Wendt  and  Butts  (1962)  cited  five  studies  which  indicated  no 
significant  differences  in  the  Instructional  effectiveness  of  tape- 
recorded  Instruction  when  compared  to  conventional  lecture  presentation, 
and  one  study  with  differences  in  favor  of  conventional  lecture  presen¬ 
tations.  Popham  (1961;  1962)  used  tape-recorded  lectures  augmented  by 
brief  Instructor-led  discussion  groups  In  both  graduate  and  under¬ 
graduate  education  courses.  When  the  experimental  graup  was  compared 
with  a  group  taught  by  conventtonaMecture  and  discussion  methods, 
no  significant  differences  In  achievement  were  found.  Meene*  Hannwn, 
Klingensmith,  and  Nord  (1969)  confirmed  and  extended  Popham*s  previous 
findings.  Tape-recorded  lectures,  not  au^nented  by  group  discussions, 
were  reported  to  be  as  effective  as  traditional  lecture  pi'esentatlon  In 
supplying  Information  to  college  undergraduates.  The  study  also  Indi¬ 
cated  that  only  S  students  In  the  taped-lecture  groups  dropped  the  course 
as  compared  to  58  students  In  the  conventional  lecture  groups.  From  the 
study*  It  Is  Impossible  to  determine  whether  to  attribute  the  lower  num¬ 
ber  of  dropouts  In  the  course  to  the  method  (e.g.,  taped  lectures)  selec¬ 
tion  process  (e.g.,  volunteers)  or  possibly  some  unidentified  variable. 
The  students*  reactions  to  the  taped  lectures  were  described  as  generally 
favorable. 
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Innovations  in  Education 

Rogers'  (1962)  observation  that  the  average  Aaierlcan  school  lags 
2S  years  behind  the  best  practice  ms  supported  by  studies  of  educa^- 
tional  change  conducted  by  tiort  (1964}  and  Mort  and  Cornell  (1941)<  Iht 
early  periods  of  educational  change  are  characterized  by  activities 
which  serve  to  sensitize  and  Motivate  Individuals  and  groups  to  dtange. 
The  early  period  of  educational  change  Is  thought  to  last  approximately 
15  years  and  results  in  from  only  3S  to  4S  of  the  school  systems  having 
accepted  a  new  practice  (Hort,  1964). 

^tderton's  (1967)  inscription  of  ^  technology  of  listructlon* 
Hechner's  (1966)  description  of  benavloral  technology*  and  KPrsh's 
(1965)  description  of  a  process  for  programning  classroom  Instruction 
are  all  very  similar  to  the  system  models  for  design  of  Instruction 
which  are  currently  being  advocated.  Because  of  the  great  similarity 
between  the  process  of  designing  progranmed  Instruction  and  the  current 
system  models  for  design  of  instruction*  It  seems  proper  to  use  an 
article  by  Skinner  (1954)  to  mark  the  Inception  of  the  use  of  system 
models  for  design  of  Instruction.  Gy  using  1954  as  the  starting  point 
for  the  use  of  system  models  for  the  design  of  Instruction*  and  applying 
Hort's  (1964)  analysis  of  innovation  In  education.  It  would  be  evident 
that  the  first  phase  of  change  (e.g.,  15  years)  should  be  ending  at  this 
time.  The  second  phase  of  approximately  20  years  should  be  marked 
by  a  rapidly  accelerated  rate  of  acceptance  and  application  of  the  new 
practice  with  approximately  7SX  more  schools  having  accepted  the  new 
practice.  The  final  phase  of  approximately  1 5; years  Is. observed ito 


34 


elapse  before  a  new  idea  or  practice  is  accepted  by  almost  100% 
of  the  schools  (Mort»  1964). 

By  projecting  tha  model  of  change  presented  for  the  system  models 
for  design  of  instruction,  the  second  phase  shojld  end  approximately 
1990  and  the  third  phase  after  the  close  of  the  twentieth  century. 
O'Toole  (1968)  stated  that  the  use  of  the  "systems  approach"  and  systems 
analysis  techniques  in  the  design  and  development  of  instructional  inno¬ 
vations,  use  of  technology  to  improve  instruction  and  accommodate  more 
students  and  requiring  students  to  assume  a  greater  responsibility  for 
their  education  represent  the  three  major  trends  of  great  significance 
for  higher  education. 

Moore  and  Heald  (1968)  concluded  that  innovation  and  change  have 
high  and  almost  automatic  credibility  today  because  educators  accept  the 
equation  "change  *  good."  Although  resistance  to  change  is  currently 
viewed  as  an  unpopular  stance,  there  are  seven  conditions  under  which  it 
is  considered  legitimate  to  resist  change: 


1.  When  the  proposed  change,  however  attractive,  is  unalterably 
out  of  phase  with  exising  sequences. 

2.  Whan  the  proposed  change  takes  the  school  system  past  the 
"point  of  no  return"  (PNR)  without  assurance  that  the  new 
conditions  beyond  PNR  will  be  better  than  the  old. 

3.  When  the  attractiveness  of  proposed  change  is  a  function  of 
an  attractive  but  dissimilar  environment. 

4.  When  the  economic  consequence  to  existing  programs  is  out  of 
proportion  to  the  potential  good  to  be  derived  from  the 
proposed  change. 

5.  When  the  success  of  a  proposed  change  is  dependent  upon 
specialized  personnel  resources  unavailable  to  the  potential 
adopter. 
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6.  When  potential  physical,  psychological,  or  academic  dangers 
of  great  magnitude  cannot  oe  tested  under  controlled  conditions. 


7.  When  tha  proposed  change  will  preclude  or  prolong  a  better 
conceived  and  more  permanent  solution  to  the  problem  under 
consideration  (p.  117-118). 

Mayhew  {1967),  in  a  Southern  Regiohal  Education  Board  monograph, 
directly  addressed  the  problem  of  implementing  innovative  changes  in 
the  instructional  programs  of  instHutions  of  higher  education.  The 
techniques  and  mechanisms  for  change  discussed  were  not  highly  struc- 
tured  models  for  change  but  rather  very  generalized  statements  such  as: 

1.  Innovation  and  change  are  not  likely  to  come  about  unless 
the  need  for  them  is  clearly  perceived  (p.  36). 

2.  Perhaps  the  most  important  element  in  effecting  changed 
practice  on  the  part  of  individual  professors  is  to 
contrive  to  have  them  become  personally  involved  in  a 
movement  which  makes  explicit  to  them  the  importance  of 
teaching  ...  (p.  29). 

3.  If  a  teacher  can  be  associated  with  others  in  some  joint 
undertaking  he  gains  considerable  strength  from  this  and 
seems  willing  to  depart  from  orthodoxy  (p.  30). 

4.  .  .  .  nothing  succeeds  like  success  (p.  40). 

The  discussion  of  change  strategy  was  concluded  by  Mayhew  (1967)  listing 
the  following  six  general  principles  related  to  the  implementation  of 
change  in  the  instructional  programs  of  institutions  of  higher  education: 

1.  .  .  .  vigorous,  strong,  and  occasionally  ruthless  administra¬ 
tive  power  is  necessary  (p.  44). 

2.  .  .  .all  human  beings,  including  faculty  inembers,  are 
sufficiently  venal  so  that  it  is  possible  to  purchase 
interest  or  to  purchase  loyalty  (p.  45). 

3.  .  .  .  leadership  for  innovation  and  change  can  be  exerted 
by  almost  anyone  who  begins  to  make  the  motions  of  a  leader 
(p.  45). 
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4.  .  .  .  Improvement  requires  time  (p.  46). 

5.  .  .  .  Innovation  1$  likely  to  be  encouraMd  If  the  Institution 
develops  a  sufficiently  refined  system  of  cost  accounting 

so  that  actual  costs  of  Instruction*  as  presently  performed* 
can  be  revealed  (p.  46). 

6.  In  some  way  or  other*  the  officer  or  agency  for  change 
must  build  Into  the  planning  a  provision  for  alleviating 
faculty  anxiety  and  insecurity  (p.  47). 


Summary 

The  survey  of  the  literature  clearly  Indicated: 

1.  New  modele  for  preservice  programs  of  teacher  education  have 
been  designed  (Ehgbretson*  1969). 

2.  Individualization  was  a  common  element  of  the  models  for 
preservice  programs  of  teacher  education  (Monson*  1969). 

3.  Behavioral  objectives  stated  for  the  models  were  comparable 
with  present  course  structure  In  the  better  teacher  training  Institu¬ 
tions  (Sllberman  &  Kool,  1969). 

4.  Program  support  systems  to  design*  construct  and  test  Instruc¬ 
tional  modules  were  omltti'd  In  most  of  the  models  (Sllberman  &  Kool, 
1969). 

5.  System  models  for  design  of  Instruction  have  been  shown  to 
result  In  Instructional  systems  which  significantly  Improve  learner 
performance  (Hagerty*  1970;  0.  Narkle*  1967;  Short,  et  al.,  1968). 

6.  Criterion-referenced  measures  are  applicable  to  programs  of 
teacher  education  (Garvin*  1970). 
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7.  Evaluation  techniques  for  use  with  mastery  models  and  criterion- 
referenced  measures  need  to  be  developed  and  tested  under  actual  instruc 
tional  conditions. 

8.  Technology  could  be  utilized  more  effectively  to  support  instruc 
tional  systems  (Morphet  &  Jesser,  1968). 


CHAPTER  III 


SYSTEM  MODELS  FOR  DESIGN  OF  INSTRUCTION 

Three  system  models  for  design  of  instruction  were  analyzed 
and  compared  on  the  basis  of  activities  required  to  implement  the 
model.  The  models  selected  were:  (1)  "A  General  Strategy  for  the 
Design  of  New  Multimedia  Courses  of  Instruction,"  based  on  Briggs 
(1968,  1970)  and  Briggs,  et  al.,  (1967),  (2)  "Model  for  the  System 
Approach  to  Education,"  based  on  Dick's  (1969)  and  Hagerty's  (1970) 
fui^ther  delineation  of  the  steps  involved,  and  (3)  the  Production, 
Implementation,  Evaluation  and  Revision  of  Instructional  Modules  (PIERIM) 
model . 


Explication  of  the  PIERIM  Model 

The  PIERIM  model  for  design  of  instruction  involves  four  phases 
of  systematic  interaction  between  an  instructor  (e.g.,  subject  mattei^ 
expert)  and  an  educational  technologist  for  the  purpose  of  designing 
self-instructional  modules  (see  Figure  2).  The  following  explication 
of  the  PIERIM  model  is  divided  into  the  four  phases: 

Phase  l"Design  of  Instructional  Modules 

Phase  2— Implementation  and  Evaluation  of  Instructional  Modules 
in  a  Conventional  Classroom  Environment 

Phase  3“Revision  of  Instructional  Modules  and  Tests 

Phase  4— Implementation  and  Evaluation  of  Instructional 
Modules  in  a  Self-Instruction  Environment. 
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PHASE  1 

DESIGN  OF  INSTRUCTIONAL  MODULES 
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production,  implementation,  evaluation, 
I  of  instructional  modules  (PIERIM) 
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Phase  I—Deslgn  of  Instructional 
[(o^uTes 

The  design  of  Instructional  modules  is  accomplished  through  the 
planned  interaction  between  an  instructor  and  educational  technologist 
for  the  purpose  of  specifying:  (1)  content  classification/identifier, 

(2)  purpose,  (3)  behavioral  objective,  (4)  test  situation,  and  (5) 
resources  for  each  of  a  set  of  observable  learner  behaviors  that  are 
expected  to  have  occurred  upon  completion  of  a  traditional  series  of 
lecture  presentations  and  classroom  activities.  The  activities  required 
are  identified  for  the  Instructor  (I)  and  Educational  Technologist  (T), 
as: 

1.  (I)  Select  the  content  area. 

2.  (I)  Write  a  detailed  outline  of  the  content  area. 

3.  (I)  Verbally  state  learner  competencies  to  be  developed. 

4.  (T)  Write  behavioral  objectives. 

5.  (T)  Write  sample  test  item  for  each  behavioral  objective. 

6.  (I)  Verbally  state  the  purpose  for  the  learner  achieving 

' each  behavioral  objective. 

7.  (I)  List  resources  for  each  behavioral  objective. 

8.  (I)  Catejorize  each  behavioral  objective  with  the  content 

outline, 

9.  (T)  Write  instructional  module  for  each  behavioral  objective. 

10.  (T)  Write  evaluation  instruments. 

The  instructor,  within  the  PIERIM  model,  has  the  final  authority  with 
respect  to  changes  in  Instructional  modules  and/or  test  items  The 
educational  technologist  is  expected  to  ask  the  instructor  for  the 
basis  on  which  his  decisions  art  made  but  th(  ultimate  responsibility 
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and  authority  within  the  model  is  assigned  to  the  instructor.  A  flow 
chart  (see  Figure  3)  depicts  the  activities  associated  with  Phase  1-- 
Design  of  Instructional  Modules  of  the  PIERIM  model. 

Phase  2--Implementation  and  Evaluation 
Of  Instructional  Modules  in  a  Conven~ 
tional  Classroom  Environment 

The  implementation  of  the  instructional  modules  in  a  conventional 
classroom  environment  is  designed  to  provide  an  opportunity  for:  (1) 
tape  recording  of  the  instructor's  lectures,  (2)  subjective  learner 
evaluation  of  the  clarity  and  congruence  among  the  description  of 
purpose,  behavioral  objective,  and  test  situation  for  each  instructional 
module,  (3)  refinement  of  resources  by  the  learners  through  the  inclu¬ 
sion  of  chapter  and  page  numbers,  (4)  subjective  learner  evaluation  of 
resources,  and  (5)  identification,  by  the  learners,  of  additional 
resources  related  specifically  to  the  instructional  modules.  The  eval¬ 
uation  of  the  instructional  modules  represents  a  formative  evaluation 
which  is  based  on  the  learners'  pre-  and  posttest  performance.  The 
activities  required  during  Phase  2  are  identified  as: 

11.  (I)  Administer  pretest. 

12.  (T)  Prepare  summary  of  learner  performance  on  pretest. 

13.  (I)  Implement  instructional  modules  in  a  conventional 

classroom  environment. 

14.  (I)  Tape  record  the  lectures  and  classroom  activities. 

15.  (1)  Administer  posttest. 

16.  (T)  Prepare  summary  of  learner  performance  on  posttest. 
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V'-!!  Instructional 

Module^s 

e  r»‘.’S’on  of  each  instructional  irodule  is  based  on  the 
learneff '  (1)  subjective  evaluation  of  the  clarity  and  congruence  among 
the  stated:  ourpose,  behavioral  objective  and  test  situation,  (2) 
perforc'ance  or.  evaluation  exercises,  and  (3)  identification  of  additional 
resources  and/or  refinement  of  existing  resources.  The  activities 
required  du^’nq  Phase  3  are  identified  as: 

17.  (Tl  Revise  instructional  modules. 

18.  (T)  Revise  evaluation  exercises. 

^hase  4--Iri)p lenientation  and  Evaluation 
Of  Instruct ipnai  lodules  in  a  Self- 
Instruction  Environment 

^his  nrase  of  the  PIERIM  model  represents  the  first  implementa¬ 
tion  of  the  instructional  modules  in  a  self-instruction  environment  for 
which  t^ey  ar'p  designed.  The  time  and  instructional  modules  represent 
fixed  variables  in  that  each  learner  receives  each  instructional  module 
and  is  expected  to  complete  them  within  a  fixed  amount  of  time 
prior  to  the  pOSttCSt. 

The  evaluation  represents  a  formative  evaluation  of  the  instruc¬ 
tional  modules  which  Is  based  on  the  learners*  performance  on  the  pre- 
and  posttest.  A  oortion  of  the  instructional  modules  (t.e.,  those 
evaluated  by  multiple  choice  test  items)  are  implemented  in  a  self- 
instruction  environment  »nd  the  remainder  of  the  instructional  module 
(i.e.,  those  evaluated  by  the  instructor)  are  implemented  in  a  conven¬ 
tional  env'»onr«ot.  Tor  each  instructional  module,  if  the  learners* 
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performance  is  judged  to  be  satisfactory  (i.e.,  a  specified  proportion 
of  the  learners  achieve  the  specified  standard  of  performance) ,  than  the 
instructional  module  is  not  revised  a  second  time.  For  each  instruc¬ 
tional  module  judged  to  be  unsatisfactory,  the  cycle  of  revision,  imple¬ 
mentation  in  a  sel  i'-instruction  environment  and  evaluation  is  repeated 
until  the  specified  proportion  of  the  learners  achieve  the  standard  of 
performance.  The  activities  required  during  Phase  4  are  identified  as: 

19.  (I)  Administer  pretest. 

20.  (T)  Prepare  summary  of  learner  performance  on  pretest. 

21.  (I)  Implement  instructional  modules  in  self-instruction 

environment. 

22.  (I)  Administer  posttest. 

23.  (T)  Prepare  summary  of  learner  performance  on  posttest 

24.  (I)  Implement  instructional  modules,  evaluated  by 

Instructor,  in  conventional  classroom  environment. 

25.  Repeat  steps  17  through  24  until  each  instructional  module 
achieves  the  standard  set  for  the  instructional  system. 


Anal^is  of  Three  System  Models 
fer  Oesiqn  of  Instruction 

Each  of  the  system  models  for  design  of  instructioi.  {i.e.,  Briggs, 
Dick  and  PICRIM)  are  analyaed  in  terms  of:  (1)  assumptions  upon  which 
the  model  is  based,  (2)  personnel  required  to  implement  the  model,  (3) 
Intended  level  of  application,  (4)  level  of  analysis,  and  (S)  activi¬ 
ties  required  to  be  completed  in  the  Implementation  of  the  model. 
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PIERIM  Model 


Assumptions 

The  PIERIM  model  for  design  of  instruction  is  predicated  upon 
five  assumptions: 

1.  Performance-based  teacher  training  programs  require  a  variety 
of  instructional  systems,  one  of  which  could  be  concerned  pri¬ 
marily  with  enabling  learners  to  master  relevant  cognitive 
skills. 

2.  A  significant  portion  of  the  cognitive  skills  which  comprise 
existing  preservice  teacher  training  programs  is  applicable 
in  performance-based  teacher  training  programs. 

3.  The  instructor  is  a  subject  matter  expert. 

4.  The  instructor  is  primarily  a  selector  rather  than  a  designer 
of  instruction. 

5.  Existing  system  models  for  design  of  instruction  are  best 
suited  for  design  of  instruction  rather  than  selection  of 
instruction. 

Personnel  Required 

The  model  specifically  states  that  an  instructor  who  is  con¬ 
sidered  to  be  a  subject  matter  expert  and  an  educational  technologist 
are  required  to  implement  the  model.  Each  activity  associated  with  the 
implementation  of  the  model  specifies  which  of  the  two  persons  is  respon¬ 
sible  for  the  accomplishment  of  the  activity. 

The  educational  technologist  is  assumed  to  have  demonstrated  his 
competence  by  either  actually  implementing  a  system  model  for  design  of 
instruction  or  having  completed  a  course  related  to  the  application  of 
system  models  for  design  of  instruction  with  a  grade  of  either  A  or  B. 
Gagne  (1969)  identified  five  categories  of  intellectual  skills  which  are 
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desirable  for  an  educational  technologist:  analyzing  learning  outcomes, 
techniques  of  measurement  of  outcomes,  constructing  empiricai  tests  of 
learning  outcomes,  statistical  competence,  and  communication  skills. 

The  PIERIM  model  operationally  defines  the  set  of  activities  the  edu¬ 
cational  technologist  must  perform  to  implement  the  model.  These  skills 
can  be  primarily  categorized  under  Gagne's  scheme: 

1.  Communication  skills--The  educational  technologist  is  required 
to  communicate  with  the  instructor,  interpret  the  instructor's  verbal 
description  of  the  objectives  of  the  course  and  translate  the  course 
objectives  into  a  set  of  instructional  modules. 

2.  Techniques  of  measurement  of  outcomes--The  educational  tech¬ 
nologist  is  required  to  design  methods  of  evaluating  the  desired  out¬ 
comes  of  the  course 

3.  Constructing  empirical  test  of  learning  outcomes--the  educa¬ 
tional  technologist  is  required  to  construct/select  the  actual  test 
situations  to  evaluate  the  learning  outcomes  specified. 

4.  Statistical  competence--the  educational  technologist  is  required 
to  analyze  the  learner's  performance  and  interpret  the  results  to  the 
instructor.  Coiiipetcnc.,  in  pci u.ucur’c  and  non  paramcti  ic  stat^'^tics  is 
desirable. 

In  addition  to  the  categories  of  intellectual  skills  described,  the 
educational  technologist  is  required  to  be  competent  in  the  production, 
testing,  and  modification  of  computer  programs  written  in  FORTRAN  IV 
programming  language. 
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Level  of  Application 

The  PIERIM  model  is  designed  specifically  for  application  to 
existing  courses  of  a  preservice  program  of  teacher  education.  The 
maximum  level  at  which  the  PIERIM  model  is  designed  to  operate  is  the 
course  level.  Any  course,  or  portion  thereof,  may  be  selected  and  the 
PIERIM  model  appropriately  applied  to  the  unit  of  instruction  selected. 

Level  of  Analysis 

Instructional  modules  are  defined  as  statements  which  contain 
the  following  i.^formation:  (1)  content  classification/identifier,  (2) 
purpose, (3)  behavioral  objective,  (4)  test  situation,  and  (5)  resources 
(after  Esbensen,  1968).  The  instructional  module  is  the  basic  unit 
produced  through  the  application  of  the  PIERIM  model  and  the  basic  unit 
of  analysis  and  revision. 

Activities 

The  following  activities  are  required  to  implement  the  PIERIM 

model : 

1.  Select  the  content  area. 

2.  Write  a  detailed  outline  of  the  content  area. 

3.  Verbally  state  learner  competencies  to  be  developed 

4.  Write  behavioral  objectives. 

5.  Write  sample  test  item  for  each  behavioral  objective. 

6.  Verbally  state  the  purpose  for  the  learner  achieving  each 
behavioral  objective. 

7.  List  resources  for  each  behavioral  objective. 
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8.  Categorize  each  behavioral  objective  within  the  content  outline. 

9.  Write  instructional  module  for  each  behavioral  objective. 

10.  Write  evaluation  instruments. 

11.  Administer  pretest. 

12.  Prepare  summary  of  learner  performance  on  pretest. 

13.  Implement  instructional  modules  in  a  conventional  classroom 
environment. 

14.  Tape  record  the  lectures  and  classroom  activities. 

15.  Administer  posttest. 

16.  Prepare  summary  of  learner  performance  on  posttest. 

17.  Revise  instructional  modules. 

18.  Revise  evaluation  exercises. 

19.  Administer  pretest. 

20.  Prepare  summary  of  learner  performance  on  pretest. 

21.  Implement  instructional  modules  in  self-instruction  environment. 

22.  Administer  posttest. 

23.  Prepare  summary  of  learner  performance  on  posttest. 

24.  Implement  instructional  modules,  evaluated  by  instructor,  in 
a  conventional  classroom  environment. 

25.  Repeat  steps  17  through  24  until  each  instructional  module 
achieves  the  standard  set  for  the  instructional  system. 

A  General  Strategy  for  Design  of  New 
Multi -media  Coupes  o^  ^nstruct^on 

This  analysis  of  the  system  model  for  design  of  instruction  is 

based  on  Briggs  (1968,  1970}  and  Briggs,  et  a1.,  (1965).  The  model  is 

represented  in  a  flow  diagram  (see  Figure  4). 
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-A  model  for  the  design  of  Instruction  source:  L.  J.  Brig 
A  Handbook  of  Procedures  for  the  design  of  Instruction 
(Pittsburgh,  Pa.:  American  Institutes  for  Research,  1970. 
Copyright,  AIR  1970) 


Assumptions 

Briggs'  (1970)  model  is  based  on  three  assumptions: 

1  Gagne's  (1965)  types  and  conditions  of  learning. 

2.  Predesigned  instruction  is  better  than  instruction  produced 
by  selecting  from  existing  resources. 

3.  Education  and  industry  need  to  establish  a  new  relationship 
for  the  design  and  production  of  instructional  materials. 

Personnel  Required 

No  particular  specialist  is  identified  with  each  of  the  activi¬ 
ties  required  to  implement  the  model.  Individuals  have  implemented  the 
model  for  limited  segments  of  instruction.  These  individuals  have  been 
enrolled  in  a  course  in  instructional  design  taught  by  Dr.  Briggs  at 
the  Florida  State  University.  It  may  be  assumed  that  a  team  of  experts 
would  generally  be  required  to  implement  the  model  on  a  large  scale. 

Level  of  Application 

The  model  is  specifically  intended  for  the  design  of  new  multi - 
media  courses.  Maximum  reliance  on  self-instructional  materials  and 
media  is  encouraged  but  the  mpdcl  provides  for  either  individualized  or 
group  instruction.  A  special  form  of  the  model  was  used  as  the  bid 
specifications  for  the  Naval  Academy  project.  The  Naval  Academy  project 
involves  the  design  and  implementation  of  university  level  courses  in 
an  individualized  instruction  environment. 

Level  of  Analysis 

Behavioral  objectives  are  analyzed  into  subordinate  conv»tenc1es 
and  the  behavioral  objective  and  subordinate  competencies  are  sequenced 
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according  to  Gagne's  (1965)  types  of  learning.  The  media  prescriptions 
are  written  for  each  subordinate  competency. 

Activities 

1.  State  objectives  and  performance  standards. 

2.  Write  test  items  for  each  behavioral  objective. 

3.  Analyze  behavioral  objectives  for  structure  and  sequence. 

4.  Identify  assumed  entering  con^etencies. 

5.  Prepare  pretests  and  remedial  instruction. 

6.  Select  media  and  write  prescriptions. 

7.  Produce  instructional  materials  In  the  media  chosen. 

8.  Conduct  individual  or  small  group  field  tests  of  instructional 
materials,  and  revise  instructional  materials  on  the  basis  of 
learner  performance. 

9.  Implement  the  course. 

10.  Evaluate  learner  performance  and  revise  course  on  the  basis  of 
learner  performance. 

Model  for  the  Systems  /^proach  to  Education 
The  analysis  of  the  system  model  for  design  of  instruction  is 
based  on  Dick's  (1969)  and  Hagerty's  (1970)  further  delineation  of  the 
steps  involved  in  the  model.  The  model  is  represented  in  a  flow  diagram 
(see  Figure  5). 

Asstanptions 

There  are  no  assumptions  explicitly  stated  concerning  the  model. 
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Personnel  Required 

There  are  not  aoy  specific  specialists  Identified  with  each  of 
the  activities  required  to  Implement  the  model.  The  model  has  been 
Implemented  by  both  Individual  learners  and  teams  of  four  or  five 
learners  for  the  purpose  of  designing  a  limited  segment  of  programmed 
instruction.  The  materials  were  developed  as  part  of  a  graduate  course 
In  programmed  Instruction.  The  course  has  been  taught  at  The  Florida 
State  University  In  conventional  classroom  environments  and  under  a 
computer-managed  Instruction  environment. 

Level  of  Application 

The  model  was  originally  designed  to  be  Implemented  In  connec¬ 
tion  with  the  development  of  programmed  Instruction.  Although  Instruc¬ 
tional  materials  at  the  level  of  a  course  or  less  have  been  developed 
through  the  Implementation  of  the  model,  there  Is  not  a  theoretical 
limit  to  the  magnitude  of  the  educational  problem  to  which  the  model 
could  be  applied. 

Level  of  Analysis 

Terminal  performance  tasks  are  analyzed  Into  subordinate  compe¬ 
tencies.  Behavioral  objectives  are  derived  from  the  subordinate  com¬ 
petencies  and  then  the  behavioral  objectives  are  sequenced  on  the  basis 
of  a  hlerarchlally  derived  sequence. 

Activities 

1.  Identify  the  problem  or  content  area. 

2.  Analyze  the  structure  of  the  problem. 
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3.  List  entering  skills  and  knowledge  assumed  as  prerequisites, 

4  Write  behavioral  objectives  and  test  items  for  each  behavioral 
objective. 

5.  Sequence  the  behavioral  objectives. 

6.  Select  media  for  behavioral  objectives  and  produce  the  instruc¬ 
tional  materials. 

7.  Implement  the  instructional  materials. 

8.  Evaluate  learner  performance. 

9.  Revise  instructional  materials  and  any  preceding  steps  on 
the  basis  of  learner  performance. 


Comparison  of  the  System  Itodels 
for  Design  of  Instruction 


Origin  of  the  Models 

Each  of  the  three  models  evolved  from  distinct  areas  of  concern: 
Briggs'  model  from  problems  related  to  the  pre-design  of  materials  spe¬ 
cifically  for  objectives,  Dick's  model  from  problems  related  to  the 
design  of  programmed  instruction,  and  PIERIM  model  from  the  identified 
lack  of  instructional  materials  to  support  an  individualized  preservice 
teacher  training  program.  Briggs'  model  was  determined  to  be  oriented 
exclusively  to  an  analysis  of  learner  behaviors  while  Dick’s  model  and 
the  PIERIM  model  consider  learner  behavior  within  the  constraints  of 
the  content  area  selected. 

Assumptions 

Briggs'  model  and  the  PIERIM  model  are  both  based  on  specific 
assumptions.  Each  model  makes  assumptions  which  serve  as  a  rationale 
for  the  development  of  the  model.  The  PIERIM  model  also  makes  specific 
assumptions  concerning  the  role  and  academic  qualifications  of  the 
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instructor.  Briggs'  model  is  the  only  model  which  makes  any  assumptions 
related  to  learning  (i.e.,  Gagne's  types  and  conditions  of  learning). 

The  types  and  conditions  of  learning  are  then  used  as  an  integral  part 
of  Briggs'  model.  Briggs'  and  Dick's  models  are  primarily  concerned 
with  design  of  new  instructional  materials  and  the  PIERIM  model  is 
explicitly  a  selection  model  for  design  of  instruction. 

Personnel  Required 

Both  Briggs'  and  Dick's  models  have  been  implemented  by  indi¬ 
viduals  and  teams  of  experts.  The  competencies  required  to  successfully 
implement  either  model  must  be  derived  from  the  author's  description 
of  the  activities  required  to  implement  his  model.  The  PIERIH  model  is 
the  only  one  of  the  three  models  reviewed  that  is  designed  as  an  inter¬ 
active  model  with  each  activity  assigned  to  either  the  instructor  or 
the  educational  technologist. 


Level  of  Application 

Each  of  the  models  is  intended  for  application  at  the  course 
level  and  could  be  applied  to  any  subportion  of  a  course.  Briggs'  and 
Dick's  models  could  theoretically  be  applied  to  the  design  of  instruc¬ 
tional  systems  at  any  level  of  organization  and  Dick's  model  has  been 
demonstrated  to  be  applicable  to  educational  problems  other  than  design 
of  instruction.  The  PIERIM  model  is  intended  for  application  only  where 
existing  courses  are  to  be  transformed  into  instructional  modules  which 
could  be  utilized  in  a  self- instructional  environment. 
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Level  of  Analysis 

The  most  detailed  analysis  of  learner  competencies  is  explicitly 
required  by  the  Briggs  model  which  requires  media  prescriptions  at  the 
instructional  event  l^'vel  for  each  subordinate  competency  of  behavioral 
objectives.  Dick's  model  requires  the  analysis  of  subordinate  compe¬ 
tencies  for  the  stated  behavioral  objectives  but  media  selection  is 
not  explicitly  required  at  each  of  the  subordinate  competencies  level 
of  analysis.  The  PIERIM  model  does  not  require  the  analysis  of 
behavioral  objectives  into  subordinate  competencies.  The  behavioral 
objective  is  the  smallest  unit  of  analysis  and  existing  materials  are 
then  selected  which  will  hopefully  enable  the  learner  to  achieve  the 
stated  level  of  competency. 

Activities 

Each  of  the  three  models  defines  the  instructional  problem  by 
different  methods.  Briggs'  model  assumes  the  existence  of  higher  order 
educational  goals  which  are  then  analyzed  in  terms  of  learner  compe¬ 
tencies  required  to  achieve  a  higher  order  educational  goal.  Dick's 
model  explicitly  requires  the  statement  of  the  problem,  prliwrlly  in 
terms  of  the  content  to  be  developed  or  preferably  In  terms  Of  desired 
terminal  performances.  The  PIERIM  model  exfillcltly  requires  the  instruc¬ 
tor  to  specify  the  content  to  which  the  model  will  be  applied. 

Each  of  the  models  requires  the  statement  of  behavioral  objectives 
in  terms  of  observable  learner  behaviors,  and  test  Items  written  directly 
from  the  stated  behavioral  objectives. 
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The  inclusion  and/or  placement  of  the  task  analysis  differs  for 
each  of  the  models.  Briggs'  model  starts  with  the  statement  of  behavioral 
objectives  for  more  general  needs  and  then  analyzes  the  behavioral 
objectives  into  their  subordinate  competencies,  Dick's  model,  on  the 
other  hand,  starts  with  a  statement  of  the  problem  in  terms  of  desired 
terminal  performance  and  the  task  analysis  describes  the  content  in 
terms  of  learner  competencies  required  to  master  the  specified  perfor¬ 
mances.  The  learner  competencies  are  then  stated  in  terms  of  behavioral 
objectives.  The.  PIERIM  model  does  not  include  a  task  analysis  of 
either  the  content  or  learner  behaviors.  A  detailed  content  outline  is 
substituted  for  the  task  analysis. 

Briggs'  and  Dick's  models  each  require  the  specification  of  the 
entering  learner  competencies  assumed  as  a  prerequisite  for  the  course 
which  would  be  developed  utilizing  their  models.  Briggs'  model  pro¬ 
vides  for  the  development  of  tests  and  instructional  materials  for  the 
prerequisite  behaviors.  When  learners  do  not  possess  the  assumed  compe¬ 
tencies,  Briggs'  model  identifies  the  alternative  courses  of  action 
which  are  available  to  the  instructional  designer.  The  PIERIIl  model  does 
not  require  the  specification  of  the  assumed  prerequisite  learner  com¬ 
petencies  that  are  associated  with  the  instructional  materials  that  are 
developed. 

Behavioral  objectives  in  the  Briggs  model  are  sequenced  primarily 
on  the  basis  of  larger  needs  analysis  or  organization  of  the  course  into 
several  levels  of  objectives  followed  by  an  analysis  of  each  behavioral 
objective  as  to  the  type  of  learning  (Gagne,  1965)  represented  by  the 
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stated  behavioral  objectives.  A  logical  analysis  of  the  content  sequence 
is  considered  an  alternative,  but  less  desirable,  basis  on  which  to 
sequence  behavioral  objectives  in  both  Briggs'  and  Dick's  models.  The 
behavioral  objectives  are  subjectively  sequenced  by  the  instructor, 
within  the  constraints  of  the  stated  content  outline,  for  the  PIERIM 
model . 

Briggs'  model  represents  the  most  extensive  treatment  of  media 
selection  of  any  of  the  system  models  for  design  of  instruction  reviewed. 
Media  prescriptions  are  written  separately  for  each  useful  event  of  the 
subordinate  competencies.  The  ultimate  media  selection  attempts  to 
maximize  the  general  and  specific  conditions  of  learning  required  to 
achieve  a  specified  set  of  learner  competencies  with  a  minimum  amount 
of  media  changes.  Media  selection  within  Dick's  model  is  generally 
accomplished  at  the  behavioral  objective  level.  The  PIERIM  model  selects, 
from  all  existing  media,  instructional  materials  which  will  hopefully 
enable  a  learner  to  achieve  a  specified  behavioral  objective.  The  pri¬ 
mary  media  considered  are  printed  materials  and  audio  tapes. 

The  ability  to  produce  the  instructional  materials,  in  the  media 
specified  by  the  model  are  implicit  assumptions  of  both  Briggs'  and 
Dick's  models.  Most  of  the  applications  of  Dick's  model,  in  connection 
with  graduate  courses  at  The  Florida  State  University,  have  resulted  in 
the  production  of  printed  programmed  instruction.  Implementation  of 
Briggs'  model,  by  graduate  students,  has  resulted  in  the  production  of 
instructional  materials  in  the  media  prescribed  by  the  media  prescriptions. 

The  PIERIM  model  requires  the  production  of  instructional  materials  In 

\ 
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only  two  media:  printed  materials  and  audio  tapes.  An  instructional 
module  is  printed  for  each  stated  behavioral  objective.  The  Instruc¬ 
tional  Support  System  (ISS)  was  developed  specifically  to  produce  the 
printed  materials  required  to  implement  the  PIERIM  model. 

I 

The  first  formative  evaluation  of  instructional  materials  is 
accomplished  by:  individual  or  small  group  field  tests,  in  Briggs' 
model;  implementation  of  instructional  modules  in  a  conventional  class¬ 
room  environment,  in  the  PIERIM  model;  and  small  group  field  tests  of 
the  entire  unit,  in  Dick's  model.  The  instructional  materials  and/or 
any  preceding  steps  are  revised  on  the  basis  of  learner  performance. 

Each  of  the  models  subsequently  implements  the  total  course, 
under  the  actual  conditions  in  which  the  course  is  designed  to  operate. 
The  implementation-evaluation-revision  cycle  continues  until  the  desired 
level  of  learner  performance  is  achieved  by  each  element  of  the  course, 
and  the  course  in  toto.  A  cost/time  decision  can  also  be  made  to  stop 
the  process  at  a  predefined  level. 

The  major  activities  of  the  three  models  for  design  of  instruc¬ 
tion,  Briggs,  Dick,  and  PIERIM,  are  compared  (see  Table  1)  for  the 
purpose  of  identifying  potential  weaknesses  in  the  instructional  modules 
produced  as  a  result  of  implementing  the  PIERIM  model. 

Implications  for  the  PIERIM  MODEL 

The  comparison  of  Briggs'  and  Dick's  models  for  design  of  instruc¬ 
tion  with  the  PIERIH  model  Identified  each  of  the  following  as  potential 
weaknesses  in  the  instructional  modules  produced  as  a  result  of  imple¬ 
menting  the  PIERIM  model: 
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1.  The  relevance  of  the  instructional  module  cannot  exceed  the 
relevance  of  the  subject  matter  content  when  compared  to  a  specific 
teacher  behavior.  System  models  for  design  of  instruction  (Briggs, 
(1970;  Dick,  1969)  design  instruction  for  a  set  of  specific  learner 
competencies  as  contrasted  to  the  PIERIH  model  which  converts  exist¬ 
ing  course  content  into  a  set  of  instructional  modules.  The  content 
validity  of  instruction  designed  for  a  specific  teacher  competency 
using  either  Briggs'  (1970)  or  Dick's  (1969)  model  is  established  by 
the  procedures  used  to  design  the  instruction.  Procedures,  external  to 
the  PIERIM  model,  must  be  utilized  to  establish  the  content  validity 

of  the  instructional  modules  and/or  the  subject  matter  content  from 
which  the  modules  were  derived. 

2.  The  instructional  modules,  in  to to,  might  be  insufficient 
to  achieve  a  specified  learner  behavior.  Briggs'  (1970)  and  Dick's 
(1969)  models  provide  for  the  analysis  of  terminal  learner  behaviors 
into  a  set  of  subordinate  competencies  and  revision  of  instructional 
materials  and/or  any  of  the  set  of  activities  prescribed  by  the  model 
until  the  prescribed  standard  of  performance  is  achieved.  The  PIERIH 
model  could  produce  a  set  of  instructional  modules,  each  of  which  was 
capable  of  producing  the  desired  change  in  learner  behavior,  but  the 
total  set  of  instructional  modules  could  still  fail  to  achieve  a  spe¬ 
cific  learner  behavior  because  of  the  existence  of  an  unidentified 
learner  competency  not  contained  in  ^e  subject  matter  content  from 
which  the  modules  were  derived. 

3.  The  instructional  resources  that  are  matched  with  an  instruc¬ 
tional  modJe  could  be  inadequate  to  achieve  the  desired  learner 
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hahavior.  Briggs'  (1970)  and  Dick's  (1969)  models  design  Instruction 
for  a  specific  learner  behavior  and  the  deslgn-lmplementatlon-evaluatlon- 
revlslon  cycle  continues  until  the  desired  standard  of  performance  Is 
achieved.  The  PIERIM  model  is  totally  dependent  upon  existing  resources. 
The  maximum  effectiveness  of  an  instructional  module  produced  through 
the  Implementation  of  the  PIERIM  model  Is  determined  by  the  combined 
effectiveness  of  existing  resources.  The  actual  level  of  effective¬ 
ness  Is  limited  by  the  instructor's  and  educational  technologist's 
knowledge  of  the  resources  and/or  sources  of  resources  and  their  ability 
to  make  the  resources  available  to  the  learners. 

The  relevance  of  the  Instructional  modules  Is  considered  the 
most  serious  of  the  three  potential  weaknesses.  Expert  opinion  was 
chosen  as  the  method  to  be  used  In  determining  the  relevance  of  the 
instructional  modules  developed  In  the  present  study  for  a  preservice 
teacher  training  program.  Faculty  members  from  the  departments  of 
Elementary  Education  and  Educational  Research  at  The  Florida  State  Uni¬ 
versity  were  chosen  to  serve  as  the  experts. 

The  deficiencies  of  the  entire  set  of  Instructional  modules,  which 
are  implemented  In  the  self  instruction  environment,  will  be  identified 
by  thr  necessity  for  the  instructor  to  present  Instruction  related  to 
behavioral  objectives  which  have  not  previously  been  Identified.  Addi¬ 
tional  behavioral  objectives  which  the  learners  are  required  to  master 
prior  to  achieving  mastery  on  the  set  of  instructional  modules  which 
have  been  Implemented  in  the  conventional  classroom  environment  will 
subsequently  be  developed  Into  Instructional  modules. 
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The  Inadequacies  of  existing  resources  will  be  Indicated  by  the 
learners'  performance  on  the  posttest  being  approximately  equal  to  or 
less  than  their  performance  on  the  pretest.  Additional  weaknesses  of 
the  PIERIM  model  are  expected  to  become  apparent  during  the  actual 
Implementations  of  the  model. 


CHAPTER  IV 


IMPLEMENTATION  OF  THE  PIERIM  MODEL:  PHASE  1 
Introduction 

The  implementation  of  the  PIERIM  model,  for  a  portion  o^ 
an  undergraduate  elementary  education  course,  was  conducted  in  order 
to: 

1.  Determine  the  feasibility  of  implementing  the  model. 

2.  Refine  the  job  descriptions  for  the  participants  in  the 
implementation. 

3.  Estimate  the  cost  of  implementing  the  PIERIM  model  in 
terms  of  personnel  and  materials. 

4.  Evaluate  the  learners'  performance  when  utilizing  the 
instructional  modules  and  tests  produced  using  the  PIERIfl 
model . 

5.  Recommend  the  contents  of  training  materials  that  could  be 
used  to  develop  the  skills  identified  for  the  participants 
of  the  implementation. 

The  selection  of  the  course  and  instructor,  which  occurred  prior 
to  the  implementation  of  the  PIERIM  model,  are  discussed  and  then 
the  implementation  of  each  of  the  four  phases  of  the  model  is  reported 
separately.  Each  phase  of  the  PIERIM  model  is  reported  in  relation¬ 
ship  to: 

1.  Activities  prescribed  by  the  PIERIM  model. 

2.  Chronology  of  actual  events. 

3.  PIERIM  model  compared  with  the  actual  activities. 

4.  Summary  of  learner  performance  (i.e.*  Phase  2  and  Phase  4). 

5.  Cost  analysis. 
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A  restatement  of  the  activities  prescribed  by  the  PIERIM  model 
provides  the  framework  within  which  the  actual  activities,  reported 
in  the  chronology  of  events,  are  reviewed  and  evaluated. 

The  summary  of  learner  performance  for  Phase  2  and  Phase  4  of 
the  PIERIM  model  is  limited  to  the  following  descriptive  statistics 
for  the  pre-  and  posttest:  mean  and  standard  deviation  for  the 
total  test,  reliability  (i.e.,  KR-20)  of  the  total  test,  and  the 
mean  and  standard  deviation  for  each  instructional  module.  A  more 
detailed  analysis  of  the  learners'  performance  is  presented  in 
Chapter  VIII,  Instructional  Support  System. 

Cost  estimates  are  based  on  the  actual  funds  required  to  pay 
for  personnel,  on  an  hourly  rate  basis,  and  for  resources  on  an 
actual  consumption  basis.  An  overhead  rate  of  18%  is  applied  to 
the  total  cost  of  personnel  and  consumable  resources.  The  cost 
factors  are  based  on  the  salary  and  material  costs  applicable  at 
The  Florida  State  University  during  the  Spring  quarter  of  the  1969- 
1970  school  year. 

From  the  set  of  25  instructional  modules  vvhich  were  developed  as 
part  of  this  study,  an  instructional  module  (i.e..  Reliability/ 

Factors  Affecting)  was  selected  for  the  purpose  of  describing  and 
documenting  the  activities  associated  with  the  implementation  of  the 
PIERIM  model . 

Selection  of  Course 

The  selection  of  the  course  in  which  to  Implement  the  PIERIM 
model  was  influenced  by  two  artificial  constraints: 
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1.  The  course  had  to  be  offered  for  two  consecutive  quarters. 

2.  The  instructional  materials  produced  v/ould  be  applicable  to 
the  Florida  State  University  Preservice  Elementary  Teacher 
Training  Model. 

The  course  selected,  EED  405“-Classroom  Organization  and  Pupil 
Evaluation,  satisfied  both  requirements  and  was  a  required  course  for 
elementary  education  majors. 

Selection  of  Instructor 

Implementation  of  the  PIERIH  model  is  dependent  upon  obtaining 
the  cooperation  of  a  member  of  the  instructional  staff.  All  staff 
members  scheduled  to  teach  the  selected  course  were  advised  of  the 
general  nature  and  purpose  of  the  PIERIM  model.  One  of  the  five 
instructors  asked  for  more  information  concerning  the  PIERIM  model 
and  clarification  of  the  instructor's  responsibility.  The  educational 
technologist  discussed  the  PIERIM  model  and  emphasized  the  role  of 
the  instructor  and  the  nature  of  the  instructional  modules  which 
would  be  produced  as  a  result  of  implementing  the  model.  The  in¬ 
structor  studied  the  model  for  one  week,  after  which  he  stated  that 
he  did  not  wish  to  participate  in  the  study.  The  reasons  given  for 
not  participating  were  related  to  the  perceived  lack  of  compatibility 
between  the  use  of  the  instructional  modules  in  a  self-instruct’on 
environment  and  the  instructor's  style  of  teaching  (i.e.,  group 
oriented  discussion  and  activities)  and  the  extensive  use  of  the 
tape  recorder. 

A  graduate  assistant  in  the  Department  of  Elementary  Education 
was  scheduled  to  assist  one  of  the  faculty  members  in  teaching 
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the  course,  Classroom  Organization  and  Pupil  Evaluation.  A  meeting 
was  conducted  with  the  faculty  member,  graduate  assistant  and 
educational  technologist  for  the  purpose  of  describing  the  PIERIM 
model  in  greater  detail.  The  faculty  member  had  given  the  graduate 
assistant  primary  responsibility  for  developing  the  instruction 
for  that  portion  of  the  course  related  to  pupil  evaluation.  The 
graduate  assistant,  henceforth  referred  to  as  the  instructor,  and 
the  educational  technologist  discussed  the  PIERIIl  model  and  the 
role  of  the  instructor  within  the  model.  Subsequently  the  in¬ 
structor  agreed  to  participate  in  the  implementation  of  the  PIERIM 
model .  The  assumption  of  the  PIERIM  model  which  considered  the 
instructor  to  be  a  subject  matter  expert  was  not  considered  to  have 
been  violated.  This  conclusion  is  based  on  the  formal  courses, 
related  to  evaluation  and  measurement,  that  the  instructor  had  com¬ 
pleted. 

Attaining  the  cooperation  of  an  instructor  is  a  critical 
activity  in  the  implementation  of  the  PIERIM  model-  Since  the 
implementation  of  the  PIERIM  model  provides  instructional  modules 
and  related  test  items,  learner  performance  data,  and  estimates  of 
time  and  costs  required  to  implement  the  model,  this  information  can 
be  used  by  the  educational  technologist  in  obtaining  the  cooperation 
of  instructors  for  future  implementations  of  the  PIERIM  model  and  to 
revise  the  activities  prescribed  by  the  model. 
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Phase  I— Design  of  Instructional  Modules 

The  design  of  Instructional  modules  Is  accomplished  through 
the  planned  Interaction  of  an  Instructor  and  an  educational  tech¬ 
nologist  for  the  purpose  of  specifying  content  classification/identifier, 
purpose,  behavioral  objective,  test  situation,  and  resources  for 
each  of  a  set  of  observable  learner  behaviors  that  are  expected  to 
have  occurred  upon  completion  of  a  traditional  series  of  lecture 
presentations  and  classroom  activities.  The  five  categories  of 
Information  operationally  define  an  Instructional  module.  The 
Instructor,  within  the  PIERIM  model,  has  the  final  authority  with 
respect  to  changes  made  to  the  instructional  modules  and/or  test 
Items.  The  educational  technologist  Is  expected  to  ask  the  In¬ 
structor  for  the  basis  on  which  his  decisions  are  made  but  the 
ultimate  responsibility  and  authority  within  the  PIERIM  model  Is 
assigned  to  the  Instructor. 

Activities  Prescribed  by  PIERIM  Model 

The  activities  prescribed  by  the  PIERIM  model  for  Phase  1— 

Design  of  Instructional  Modules  are  identified  for  the  Instructor 
(I)  and  the  Educational  Technologist  (T)  as: 

1.  (I)  Select  the  content  area. 

2.  (I)  Write  a  detailed  outline  of  the  content  area. 

3.  (I)  Verbally  state  learner  competencies  to  be  developed. 

4.  (T)  Write  behavioral  objectives. 

5.  (T)  Write  sample  test  Item  for  each  behavioral  objective. 

6.  (I)  Verbally  state  the  purpose  for  the  learner  achieving 

each  behaxioral  objective. 
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7. 

(I) 

List  resources  for  each  behavioral  objective. 

8. 

(I) 

Categorize  each  behavioral  objective  within  the 
content  outline. 

9. 

(T) 

Write  instructional  module  for  each  behavioral 
objective. 

10. 

(T) 

Write  evaluation  instruments. 

Chronolooy  of  Events 

The  activities  related  to  Phase  1— Design  of  Instructional 
Modules  were  documented  through  the  evolution  of  written  materials 
and  the  use  of  audio  tapes  made  for  each  work  session.  The  activities 
are  reported  in  their  actual  order  of  occurrence.  Each  activity  is 
identified  as  to  date,  personnel  involved,  time  estimate  and  a  des¬ 
cription  of  the  activity.  The  time  reported  is  the  total  time  re¬ 
quired  to  develop  the  entire  set  of  25  instructional  modules. 


DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

12/12/69 

T 

1.0 

Discuss  the  PIERIM  model  with 
instructor 

1/09/70 

I  &  T 

.5 

Discuss  the  PIERIM  model  with 
instructor 

1/14/70 

I 

1.2 

Outline  content  area 

1/14/70 

I  6  T 

.4 

Conduct  Work  Session  -  1 

1/15/70 

I 

.9 

Verify  scope  of  content  area 
outline 

1/15/70 

T 

1.0 

Review  tape  of  Work  Session  -  1 
Prepare  questions  for  next  work 
session 

1/18/70 

T 

2.0 

Write  behavioral  objectives 
Write/select  test  Items  for 
behavlorsl  objectives 

Write  content  area  outline 
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DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

1/19/70 

I  &  T 

.7 

Conduct  Work  Session  -  2 

1/20/70 

T 

1.5 

Review  tape  of  Work  Session  -  2 

1/21/70 

I 

1.5 

Analyze  contents  of  required  text 

1/22/70 

I  &  T 

.7 

Conduct  Work  Session  -  3 

1/23/70 

T 

1.4 

Review  tape  of  Work  Session  -  3 
Prepare  questions  for  next  work 
session 

1/23/70 

T 

4.0 

Select  test  items  by  content  area 
from  test  item  pool 

1/26/70 

T 

3.4 

Write  behavioral  objectives 

Write/ select  test  items  to  match 
behavioral  objectives 

2/01/70 

I 

1,5 

Review  content  outline  in  terms  of 
congruence  of  behavioral  objectives, 
test  items,  and  text  material 

2/02/70 

I  &  T 

.5 

Conduct  Work  Session  -  4 

2/02/70 

T 

1.0 

Review  tape  of  Work  Session  -  4 
Prepare  questions  for  next  work 
session 

2/03/70 

T 

3.0 

Write  behavioral  objectives 

2/03/70 

T 

5.0 

Hatch,  modify  and/or  write  test 
items  for  behavioral  objectives 

2/04/70 

T 

3.5 

Write  first  draft  of  instructional 
modules 

2/04/70 

I  ft  T 

1.4 

Conduct  Work  Session  -  5 

2/04/70 

T 

2.0 

Review  tape  of  Work  Session  -  5 
Prepare  questions  for  next  work 
session 

2/04/70 

I 

1.5 

Sequence  Instructional  modules 

2/05/70 

I  ft  T 

.4 

Conduct  Work  Session  •  6 

2/05/70 

T 

1.0 

Review  tape  of  Work  Session  -  6 
Prepare  questions  for  next  v/ork 
session 
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DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

2/06/70 

T 

1.6 

Sequence  test  items  for  pretest 

2/06/70 

I 

.7 

Review  scope  and  sequence  of 
instructional  modules 

2/07/70 

T 

.3 

Proofread  pretest 

2/07/70 

T 

.8 

Write  introduction  to  unit 

2/07/70 

T 

1.3 

Design  forms  for  use  by  the 
learners 

2/09/70 

I  &  T 

.4 

Conduct  Work  Session  -  7 

2/09/70 

T 

.5 

Review  tape  of  Work  Session  -  7 

2/10/70 

I  &  T 

.5 

Conduct  Work  Session  -  8 

2/10/70 

T 

.5 

Review  tape  of  Work  Session  -  8 

The  accuracy  of  the  chronology  of  events  as  to  the  date  of  the  work 
sessions  and  the  activities  which  occurred  during  each  session  was  made 
possible  by  the  practice  of  tape  recording  each  of  the  work  sessions. 

The  tapes  of  each  work  session  were  reviewed  by  the  educational  tech- 
nologist  in  preparation  fo*-  the  subsequent  work  session. 

PIERIM  Model  Compared  with  the 
Actual  Activities 

The  sequence  of  activities  during  Phase  I  --  Design  of  Instruc¬ 
tional  iiodules  was  not  as  distinct  as  the  flow  chart  (see  Figure  3) 
would  Indicate.  Based  on  the  experience  gained  from  inplementatlon  of 
the  PIERIH  model*  one  additional  activity  was  identified  for  the 
instructor  (i.e.*  evaluate  behavioral  objectives  and  test  items).  The 
responsibility  for  categorizing  the  behavioral  objectives  within  the 
content  outline  was  assigned  to  the  educational  technologist.  The 
activities  were  determined  to  have  typically  occurred  as  shown  in  Figure  6. 


I 

I 
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I 

I 
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The  evolution  of  the  unit  outline  is  presented  (see  Appendix  A) 
in  three  stages:  1)  the  original  course  outline,  2)  an  interim  unit 
outline,  and  3)  the  final  outline  included  in  the  learners'  set  of 
instructional  materials.  The  first  edition  of  the  instructional 
module  and  test  items  which  relate  to  Reliability/ Factors  Affecting 
(see  Appendix  B)  is  representative  of  the  set  of  instructional  modules 
and  test  items  developed  as  the  result  of  implementing  Phase  I— 

Design  of  Instructional  Modules  of  the  PIERIM  rx)del.  The  set  of  instruc 
tional  materials  which  were  given  to  each  learner  included: 

1  Introduction 

1  Outline  of  Evaluation  Unit 

16  Instructional  modules  which  specify  that  multiple 
choice  test  Items  will  be  used  to  evaluate  learner 
performance 

9  Instructional  modules  which  specify  that  the  instructor 
will  evaluate  the  learners'  performance  by  either  short 
constructed  responses  or  problem  solving  exercises. 

2  Tables 

5  Forms  for  the  learners'  use. 

The  titles  of  the  25  Instructional  mooules  are  presented  In  Table  2. 

Old  examinations  and  test  Item  pools  were  determined  to  be  a 
valuable  source  of  multiple  choice  type  Items.  Examination  of  sets 
of  test  Items  which  had  been  classified  by  content  area  was  deter- 
mined  to  be  one  means  by  which  the  Instructor  defined  the  scope  of 
the  learner  competencies  to  be  developed.  Approximately  75  percent 
of  the  test  IteiK  developed  or  selected  were  ultimately  utilised  by 
the  Instructor  as  either  sample  test  Items  or  as  Items  on  the  pre- 
end  posttests.  The  pretest  consisted  of  42  multiple  choice  Items  and 
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TABLE  2. --Instructional  modules  designed  during  phase  I  and  method 
of  evaluation  specified 


INSTRUCTIONAL  MODULE  METHOD  OF  EVALUATION 

Hsr  INSTRU™ 


Pretest/Posttest  X 

Behavioral  Objectives  X 

Test  Items/Behavloral  Objectives  X 

Test  Items  X 

Percentile  Ranks  X 

Percentile  Ranks  X 

Measures  of  Central  Tendency  X 

Measures  of  Central  Tendency  X 

Standardized  Tests/Oerived  Scores  X 

Normal  Distribution  X 

Normal  Curve  X 

Standardized  Tests/Derived  Scores 

Percentlle/Stanine  X 

Frequency  Polygon  X 

Correlation  Coefficient  X 

Correlation  Coefficient  X 

Validity  X 

Reliability  X 

Reliability  X 

Standard  Error  of  Measurement  X 

Regression  Toward  the  Mean  X 

Tests,  Types  of  X 

Standardized  Tests/Norms  X 

Standardized  Tests/Sources  of  Infcrmatlon  X 

Standardized  Test/Interpretation  X 
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the  posttest  consisted  of  the  same  42  ^est  items  plus  13  additional 
items.  The  additional  test  items  were  included  on  the  posttest  for 
the  purpose  of  obtaining  additional  item  statistics.  The  extra  items 
were  used  to  replace  unsatisfactory  items  or  to  form  a  pool  of  test 
items  from  which  alternate  forms  of  the  test  could  be  constructed. 

Cost  Analysis 

The  activities  required  to  produce  30  sets  of  the  instructional 
modules  and  tests  were  to  type  and  proofread  stencils  from  hand¬ 
written  copy,  and  reproduce  and  collate  the  materials.  The  educa¬ 
tional  technologist  typed  the  stencils  and  then  three  experienced 
secretaries  were  asiced  to  estimate  the  time  required  to  type  and 
proofread  the  stencils.  The  average  of  their  estimates  (i.e.,  8 
minutes  per  stencil)  was  used  in  determining  the  cost  estimates. 

A  student  assistant  reproduced  and  collated  the  instructional  materials 
and  the  time  rep" '"ted  (i.e.,  3.5  hours)  is  the  actual  time  expended. 

The  costs  associated  with  the  implementation  of  the  PIERIM 
model  are  divided  into  two  categories: 

1.  Design  costs,  which  include  revision  of  materials. 

2.  Production  costs,  which  include  costs  related  to  the 

Instructional  Support  System. 

Based  on  the  actual  costs  associated  with  the  implementation 
of  Phase  I— Design  of  Instructional  Modules  (see  Table  3),  the 
cost  of  designing  one  instructional  module  and  test  items  was 
determined  to  be  $9.95  (i.e.,  $248.75  *  25).  The  cost  of 
producing  one  complete  set  of  the  25  Instructional  modules  and 
test  items  was  determined  to  be  $1.36  (i.e.,  $40.85  *  30). 


Description  of  cost  factor  Quanti ty  Unit  Rate  Design  cost*  Production  cost** 
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CHAPTER  V 


IMPLEMENTATION  OF  THE  PIERIM  MODEL:  PHASE  2 

The  instructional  modules  and  tests,  designed  and  produced 
during  Phase  1  of  the  PIERIM  model,  were  implemented  in  the  Evaluation 
Unit  of  EED  405--Classroom  Organization  and  Pupil  Evaluation  during  the 
Winter  quarter  of  the  1969-1970  school  year.  The  first  group  of 
learners  (i.e.,  Group  1)  to  participate  in  the  implementation  of  the 
instructional  modules  and  tests  were  elementary  education  majors.  The 
19  learners  were  in  either  their  junior  and  senior  year  and  enrolled 
in  additional  courses  which  placed  demands  upon  their  time- 

The  conventional  classroom  environment  for  this  study  is  opera¬ 
tionally  defined  by  the  following  characteristics: 

1.  The  principle  method  of  teaching  is  the  group 
lecture  and  discussion  method. 

2.  The  classes  are  scheduled  to  meet  at  a  regular 
time  and  place. 

3.  The  classes  meet  for  a  fixed  length  of  time. 

4.  The  instructor  and  a  required  textbook  are  the 
primary  resources  utilized  in  the  instructional 
program. 

The  activities  in  which  the  learners  typically  engaged  during  one  of  the 
three  weeks  of  the  implementation  of  the  instructional  materials  included 

1.  Monday,  the  learners  participated  in  a  group  seminar  in  which 
current  topics  of  interest  could  be  discussed  by  the  instructor 
and  learners. 

2.  Tuesday,  Wednesday,  and  Thursday,  the  class  met  for  group  lecture 
and  discussion  related  to  the  set  of  Instructional  modules. 
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3.  Friday,  a  he'o  sessiDn  w.is  provided  for  those  learners 
who  needed  additional  assistance  on  the  instructional 
nodules  which  were  evaluated  by  the  instructor. 

the  learners  were  not  required  to  attend  class  during  the  implemen¬ 
tation  of  the  instructional  modules  and  the  attendance  for  the  lecture/ 
discussion  classes  was  reported  to  average  from  11  to  13  of  the  19 
learners  and  the  attendance  at  the  help  sessions  was  much  lower.  The 
learners  were  required  to  complete  the  pre-  and  posttest  for  the  unit 
of  instruction. 

The  imolementation  o’'  ti)o  instructional  modules  in  a  conven¬ 
tional  classroom  environment  is  designed  to  provide  an  opportunity 
for: 

1.  Tape  recording  of  t‘'e  instructor's  lectures 

2.  Subjective  lea'-ner  evaluation  of  the  clarity  and  con¬ 
gruence  among  the  dr.sr of  nureose,  behavioral 
objective  and  tQ't  s:tiut'on  'or  each  in^truct'enal 
module. 

3.  Refinement  of  ty  the  learne*'s  ta'ou'ih  the 

inclusion  of  chaii+->-  and  page  numbers 

4.  Subjective  learner  evaludt'on  of  resources. 

5.  Identification  by  the  loaners  of  additional  resources 
related  spec’^ical’y  to  the  instructional  modules. 

The  evaluation  of  the  instruct ’onal  modu'es  represents  a  formative 

evaluation  which  is  ’ased  nn  t>e  learners’  pre-  and  Dosttest  per- 

*^ormance. 

Activities  Prescribed  by  f.lE'  p -odel 

The  actlvHies  prescr-bpd  ’-y  the  RIERIM  model  tor  Phase  2-- 
Impi ementation  and  tvaluif  rvp  of  Instructional  Modules  in  a  Con¬ 
ventional  Classroom  Env rennent  a-O  'dentifiod  for  the  Instructor  (1) 
and  the  Educational  Tec— 'oiogi  .i  {')  as 
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11. 

(I) 

Administer  pretest. 

12. 

(T) 

Prepare  summary  of  learner  performance  on  pretest 

13. 

(I) 

Implement  instructional  modules  in  a  con¬ 
ventional  classroom  envlronment. 

14. 

(I) 

Tape  record  the  lectures  and  classroom  activites. 

15. 

(I) 

Administer  posttest. 

16. 

(T) 

Prepare  summary  of  learner  performance  on  posttest 

Chronology  of  Events 

The  activities  related  to  Phase  2— 'Implementation  and  Evaluation 
of  Instructional  Modules  in  a  Conventional  Classroom  Environment  were 
documented  through  the  evolution  of  written  materials  and  the  use  of 
audio  tapes  made  of  each  work  session.  Each  activity  is  Identified  as 
to  date,  personnel  Involved,  time  estimate  and  a  description  of  the 
activity.  The  time  reported  Is  the  total  time  required  to  Implement 
the  entire  set  of  25  Instructional  modules. 


DATE 

PERSONNEL 

TIME/ HR. 

ACTIVITY 

2/10/70 

I 

l.O 

Administer  pretest 

2/10/70 

T 

.4 

Check  and  submit  answer  sheets 
to  the  test  scoring  service 

2/11/70 

T 

.5 

Pick  up  test  and  Item  analysis 

2/11/70 

I  &  T 

.5 

Set  up  and  test  tape  recorder 
for  recording  class  sessions 

2/14/70 

T 

1.5 

Summarize  learner  performance 
on  pretest  by  instructional 
module 

2/15/70 

T 

2.7 

Prepare  copy  of  pretest  per¬ 
formance  for  each  learner 

2/27/70 

I 

1.0 

Administer  posttest 
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DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

2/27/70 

T 

.5 

Check  and  submit  answer  sheets 
to  test  scoring  service 

3/02/70 

T 

.5 

Pick  up  test  and  Item  analysis 

3/02/70 

T 

1.0 

Prepare  summary  of  learner 
performance  for  posttest  using 
ISS  program  EVAL 

PIERIM  Model  Compared  with 
the  Actual  ActivitiM 

The  activities  originally  specified  for  Phase  2— Implementation 
and  Evaluation  of  Instructional  Modules  In  a  Conventional  Classroom 
Environment  were  found  to  occur  In  the  sequence  specified  by  the  PIERIM 
model.  Activity  number  14  (I.e.,  Tape  record  the  lectures  and  classroom 
activities)  was  found  to  be  an  unsatisfactory  method  of  documenting 
the  Implementation  of  the  Instructional  modules  In  a  conventional  class¬ 
room  environment.  The  quality  of  the  tapes  which  were  recorded  made 
the  tapes  unacceptable  as  resources  for  the  revised  Instructional 
modules.  An  alternative  method  of  recording  the  sequence  and  scope 
of  the  class  activities*  during  the  Implementation  of  the  Instruc¬ 
tional  modules.  Is  to  have  detailed  notes  taken  of  the  class  lectures 
and  discussions.  The  educational  technologist  Is  made  responsible 
for  this  revised  activity.  Either  a  repilar  member  of  the  class  or 
a  student  assistant  could  be  paid  for  performing  the  activity* 

The  activities  associated  witii  Phase  2— Implementation  and 
Evaluation  of  Instructional  Modules  1r.  a  Conventional  Classroom 
Environment  (Revised)  are: 

12.  (I)  Administer  pretest. 

13.  (T)  Prepare  summary  of  learner  performance  on  pretest. 
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14. 

(1) 

Implement  instructional  modules  in  a  conventional 
classroom  environment. 

15. 

(T) 

Prepare  detailed  notes  of  class  lectures  and 
activities. 

16. 

(1) 

Administer  posttest. 

17. 

(T) 

Prepare  summary  of  learner  performance  on  posttest. 

Summary  of  Learner  Performance 

The  learners'  performance  was  measured  fo<^  the  set  of  16 
instructional  modules  using  the  same  form  of  a  42  item  multiple  choice 
test  as  the  pre-  and  posttest.  The  learners  were  told  prior  to  taking 
the  pretest  that  the  same  test  would  be  used  to  evaluate  their  per¬ 
formance  at  the  end  of  the  evaluation  unit.  There  Is  no  precise  method 
of  determining  how  this  information  would  bias  the  performance  data 
At  least  two  learners  reported  that  other  members  of  the  class  had 
the  answers  for  some  of  the  test  Items  prior  to  taking  the  posttest. 

The  reliability  of  the  pretest  (KR-20  »  .60)  was  greater  than 
for  the  posttest  (KR-20  ■  .52).  The  means  and  standard  deviations 
(see  Table  4)  are  reported  for  each  of  the  16  Instructional  modules 
separately  and  for  the  total  test. 

Nine  of  the  instructional  modules  specified  that  the  learners' 
performance  would  be  evaluated  hy  the  Instructor.  The  instructor  v^as 
either  unwilling  or  unable  to  specify  the  criteria  he  used  to  evalu¬ 
ate  the  learners'  performance  on  each  o*  these  Instructional  modules. 

Cost  Analysis 

In  determining  the  personnel  costs  for  the  implementation 
of  the  Instructional  modules  and  tests  only  time  In  excess  of  regular 


TABLE  4. -'Learner  perfomance  data--Phase  2 
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class  time  Is  reported  for  the  Instructor.  The  time  required  for 
the  educational  technologist  to  design  the  Instructional  Support  System 
programs  Is  considered  a  one  time  cost  and  Is  not  Included  In  the 
cost  analysis  for  the  PIERIM  model.  The  costs  related  specifically 
to  the  development  of  the  Instructional  Support  System  are  reported 
In  Chapter  VIII. 

From  an  analysis  of  the  costs  associated  with  the  Imple¬ 
mentation  of  Phase  2  of  the  PIERIM  model  (see  Table  5)  It  Is  evident 
that  there  are  few  costs  Involved  In  the  actual  Implementation  of 
the  Instructional  modules  and  tests  In  a  conventional  classroom  en¬ 
vironment.  The  summary  report  of  learner  performance  was  prepared 
manually  for  the  pretest  and  by  the  ISS  program  EVAL  for  the  posttest. 
Comparison  of  the  cost  of  producing  the  report  by  the  two  methods  reveals 
that  the  co^tuter  produced  report  cost  16  cents  more  than  the  manually 
produced  report  when  only  consumables  are  Included  In  the  cost.  If  the 
additional  3.2  hours  of  the  educational  technologist's  time  required 
to  manually  prepare  the  report  were  Included  In  the  cost*  then  the 
computer-produced  report  Is  significantly  cheaper  than  the  manually- 
produced  report  (1.e.»  $1.11  vs.  $13.75). 


TABLE  5.--Cost  aodlysfs  for  frttase  2- -Implementation  and  evaluation  of  instructional  modules 
in  a  conventional  classroom  environment 
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CHAPTER  VI 


IMPLEMENTATION  OF  THE  PIERIM  MODEL:  PHASE  3 

The  precise  nature  of  the  activities  which  comprised  Phase  3-- 
Revision  of  Instructional  Modules  and  Tests  was  not  specified  by 
the  PIERIM  model.  In  the  original  set  of  procedures,  the  learner  was 
considered  to  be  a  major  contributor  to  the  revision  of  the  instruc- 
tional  modules.  The  revision  of  each  instructional  module  is  based 
on  the  learners'  subjective  evaluation  of  the  clarity  and  congruence 
among  the  stated  purpose,  behavioral  objective  and  test  situation, 
performance  on  evaluation  exercises,  and  identification  of  additional 
resources  and/or  refinement  of  existing  resources. 

Activities  Prescribed  by  PIERIM  Model 

The  activities  prescribed  by  the  PIERIM  model  for  Phase  3— 
Revision  of  Instructional  Modules  and  Tests  are  identified  for  the 
Instructor  (I)  and  the  Educational  Technologist  (T)  as: 

17.  (T)  Revise  instructional  modules. 

18.  (T)  Revise  evaluation  exercises. 

ChrwoloQV  of  Events 

The  activities  related  to  Phase  3»RevUlon  of  Instructional 
Nodules  and  Tests  were  documented  th»t>u9h  the  evolution  of  written 
materials  end  the  use  of  audio  tapes  made  of  each  work  session.  Each 
activity  is  identified  as  to  date,  personnel  involved,  time  estimate 
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and  a  description  of  the  activity.  The  time  reported  is  the  total 
time  required  to  revise  the  entire  set  of  25  instructional  modules. 


DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

2/12/70 

I  &  T 

.4 

Conduct  Work  Session  -  9 

Discuss  methods  of  summarizing 
learner  performance  and  the  use 
of  additional  resources. 

2/12/70 

T 

.7 

Review  tape  of  Work  Session  -  9 

2/12/70 

T 

1.0 

Review  tape  of  class  lecture 

2/13/70 

T 

1.0 

Re  -'iew  tape  of  class  lecture 

2/17/70 

I  &  T 

.3 

Conduct  Work  Session  -  10 

Request  for  a  summary  of  the 
number  of  learners  achieving 
the  standard  of  performance  for 
each  instructional  module  made  by 
the  instructor. 

Discuss  learner  reaction  to  the 
summary  of  performance  report 
Identify  specific  weaknesses  of 
the  instructional  modules  which 
had  been  taught. 

2/19/70 

T 

1.5 

Select  additional  library  books 
related  to  evaluation 

2/21/70 

T 

2.0 

Write  additional  test  items  for 
the  posttest 

2/24/70 

I  &  T 

.3 

Conduct  Work  Session  -  11 

Identify  specific  weaknesses  of 
the  instructional  modules  which 
had  been  taught. 

2/25/70 

T 

1,0 

Review  tape  of  Work  Session  -  10 
Review  tape  of  Work  Session  -  11 

2/26/70 

I  &  T 

.4 

Conduct  Work  Session  >  12 

Identify  specific  weaknesses  of 
the  instructional  modules  which 
had  been  taught. 
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DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

2/28/70 

T 

.5 

Review  tape  of  Work  Session  -  12 

3/09/70 

1  &  T 

.4 

Conduct  Work  Session  -  13 

Review  posttest  results 

Discuss  remaining  instruc¬ 
tional  modules 

3/09/70 

T 

7.5 

Listen  to  tapes  of  remaining 
classroom  sessions.  Only  tapes 
from  4  class  sessions  could  be 
interpreted. 

3/10/70 

T 

.9 

Prepare  sumnary  of  learner 
performance  on  pre-  and  posttest 
for  instructor. 

3/17/70 

T 

2.5 

Review  and  summarize  learner 
performance  by  instructional 
module 

3/18/70 

I 

1.8 

Review  summary  of  learner  per¬ 
formance  , 

3/19/70  - 

■■  ■'/  T 

.5  ^ 

Review  tajje  of  Work  Session  -  13 

3/20/70 

I  &  T 

4.5 

Conduct  Work  Session  -  14 

Complete  srecification  of  revisions 
required  for  instructional  modules 
and  test  items 

3/21/70 

T 

1.5 

Review  tape  of  Work  Session  -  14 

3/25/70 

T 

2.0 

Select  resource  books  from  library 

3/26/70 

T 

5.5 

'ilatch  resources  with  instructional 
tiwdules 

4/01/70 

I 

1.0 

Arrange  for  use  of  library 

fdcil  ities 
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DATE 

PERSONNEL 

TIME/ HR. 

ACTIVITY 

4/02/70 

I 

20.0* 

Prepare  audio  tapes  for  use  as 
resources  with  the  instructional 
modules 

4/15/70 

I  &  T 

.8 

Conduct  Work  Session  -  15 

Review  computer  printed  copy  of 
instructional  modules  and  tests. 

♦Time  is  a  summary  of  times  for  5  days. 


PIERIM  Model  Compared  with 
the  Actual. Activities 


The  activities  associated  with  Phase  3— Revision  of  Instructional 
Jiodules  and  Tests  began  immediately  upon  implementation  of  the  instruc¬ 
tional  modules.  The  instructor  was  asked  to  respond  subjectively  to  the 
relative  strengths  and/or  weaknesses  of  the  instructional  modules  immedi¬ 
ately  after  the  instructional  module  had  been  taught  in  the  conventional 
classroom  environment.  The  instructor  made  notes  directly  on  his  copy 
of  the  instructional  modules,  and  the  notes  were  used  to  suggest  needed 
revisions  to  the  instructional  modules.  The  learners'  pre-  and  posttest 
performance  data  was  analyzed  and  the  following  general  strategy  for 
revision  of  the  instructional  modules  and  test  items  was  identified; 

(T)  Identify  changes  which  affect  all  of  the  instructional 
modules. 

(I)  Specify  desired  changes  to  instructional  modules  based 
on  the  implementation  of  the  module. 

(T)  Identify  instructional  modules  with  large  numbers  of 
learners  in  an  unlearned  state  on  both  the  pre-  and 
posttest. 

(T)  Identify  test  items  with  low  proportion  of  learners 
passing  the  item  on  the  posttest. 

(T)  Identify  test  items  which  deviate  from  the  other  test 
items  for  the  instructional  module  either  on  proportion 
of  learners  passing  the  item  or  number  of  learners  who 
were  in  an  unlearned  state  on  both  the  pre-  and  posttest. 
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Library  books  and  audio  tapes,  which  were  utilized  as  additional 
resources  for  the  Instructional  modules,  were  available  for  use  In  the 
university  library.  The  educational  technologist  assumed  the  primary 
responsibility  for  Identifying  additional  library  books  which  were 
related  to  the  evaluation  unit.  The  card  catalog  was  utilized  to 
Identify  the  library  classification  number  associated  with  books  in 
the  area  of  educational  testing  and  measurement.  The  books  were  reviewed 
by  checking  the  set  of  key  words  used  to  identify  the  content  classifi¬ 
cation  of  each  Instructional  module  against  the  Index  of  the  book. 
Specific  pages  of  each  resource  were  matched  with  the  appropriate 
instructional  module.  The  Instructor  deleted  any  resource  which  he 
fell}  was  Inappropriate,  and  his  main  reasons  for  rejecting  resources 
were  (1)  the  resource  did  not  relate  directly  to  the  stated  behavioral 
objective  and/or  (2)  the  terminology  and  methods  used  to  develop  the 
topic  were  judged  Inappropriate  for  the  Intended  population  of  the 
learners. 

The  audio  tapes  of  the  Instructor's  lectures  had  originally  been 
identified  as  a  resource  for  the  revised  Instructional  modules.  Either 
human  error  or  equipment  failure  resulted  In  only  four  of  the  nine 
lectures  being  recorded.  The  quality  of  the  tapes  which  were  recorded 
was  judged  to  be  of  such  poor  quality  as  to  question  the  feasibility  of 
retaining  the  recording  of  classroom  lectures  as  an  activity  associated 
with  the  PIERIM  model.  Three  alternative  activities  which  occurred  and 
resulted  In  the  production  of  audio  tapes  used  as  resources  were: 
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(I)  Review  class  notes  and  additional  library  resources 
for  each  Instructional  module. 

(I)  Prepare  summary  for  each  Instructional  module. 

(I)  Prepare  an  audio  tape  of  the  summary  for  each 
Instructional  module. 

The  Instructor  reported  that  approximately  20  hours  were  required  to 
prepare  the  set  of  11  audio  tapes.  Each  tape  was  from  three  to 
eight  minutes  In  length.  A  transcript  of  the  audio  tape  for  relia¬ 
bility  (see  Appendix  C)  Is  representative  of  the  set  of  tapes  used 
as  resources  for  the  instructional  modules.  The  bibliography  for 
the  set  of  revised  Instructional  modules  (see  Figure  7)  lists  all 
of  the  resources  which  were  related  to  the  set  of  Instructional 
modules. 

The  format  of  the  Instructional  moduHes  was  changed  and 
the  following  three  changes  were  Incorporated  Into  each  instructional 
module: 

1.  Unfamiliar  terminology,  such  as  "productive  curri¬ 
culum  embedded  evaluation  exercises,"' was  replaced 
with  "open  book  written  assignment  to  be  evaluated 
by  the  Instructor." 

2.  Correct  answers  for  each  sample  test  Item  were 
Indicated. 

3.  Pages  of  the  Instructional  materials  were  numbered 
consecutively. 

The  headings  of  the  Instructional  modules  were  revised  to  include 
the  following;  content  classification,  purpose,  behavioral  objective, 
sample  test  Item  (If  the  learner's  performance  Is  evaluated  by  objec¬ 
tive  test  Items),  or  test  situation  (If  the  learner's  performance  Is 
evaluated  by  the  Instructor),  and  resources. 


Fig,  7. --Bibliography  for  revised  Instruction  modules 
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(see  Table  6)  include  any  changes,  other  than  correction  of  spelling, 
which  were  made  to  an  Instructional  module.  Revisions  for  each 
Instructional  module  are  reported  as  to  the  subheading  of  the  module 
under  which  the  change  was  made.  The  revisions  made  to  the  test 
Items  Included  the  following:  three  Items  were  replaced,  the  stems 
of  four  Items  were  revised,  and  seven  Items  required  changes  to 
one  or  more  alternatives.  The  number  of  changes  required  for  the 
set  of  Instructional  modules  and  test  Items  dictated  that  the  entire 
production  process  be  repeated.  Rather  than  repeating  the  manual 
production  cycle,  the  ISS  program  IMPROD  and  TEST  were  developed 
to  produce  the  Instructional  modules  and  tests.  These  programs  and 
the  total  ISS  are  discussed  at  some  length  In  Chapter  VIII. 

The  activities  associated  with  Phase  3>-Rev1s1on  of  Instruc¬ 
tional  Modules  and  Tests  (Revised)  are; 

18.  (T)  Identify  changes  which  affect  all  of  the  instructional 

Modules. 

19.  (I)  Specify  changes  to  be  made  to  Instructional 

Modules  and/or  test  items  based  on  a  review  of 

learner  performance  data. 

20.  (T)  Identify  additional  resources  for  the  Instructional 

ftodules. 

21.  (T)  Match  resources  to  each  Individual  Instructional 

Module. 

22-  (I)  Review  class  notes  and  additional  resources  for 

each  Instructional  Module. 

23.  (I)  Prepare  a  summary  for  each  Instructional  Module. 

24.  (I)  Prepare  an  audio  tape  of  the  summary  for  each 

Instructional  Module. 

25.  (T)  Revise  Instructional  Modules. 

26.  (T)  Revise  evaluation  exercises. 
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TABLE  6.”Rev+s1tms--instrt»ct4orMiT'WOda'leS' 


INSTRUCTIONAL  MODULE 

CATEGORY 

CHANGED 

1 

2 

3 

4 

5 

Pretest/Posttest 

X 

X 

X 

X 

2 

Behavioral  Objectives 

X 

4 

Test  Items 

4 

Percentile  Ranks 

3 

Measures  of  Central  Tendency 

6 

Normal  Distribution 

X 

5 

Normal  Curve 

4 

Correlation  Coefficient 

4 

Correl atl on/Scatter  D1 agram 

X 

2 

Validity  , 

5 

Rel labll 1 ty/Factors  Affecti ng 

Rel labll Ity/ Interpretation 

Standard  trror  of  Measurement 

X 

X 

X 

5 

X 

X 

7 

5 

Types  bf  Tests 

X 

X 

X 

4 

Test  Norms/ Intel  1 1 gence  Quotl ent 

X 

X 

X 

4 

Standardised  Test  Information 

X 

X 

2 

Test  Items/Behavloral  Objectives 

X 

X 

3 

Percentile  Rank 

X 

2 

Measures  of  Central  Tendency 

X 

X 

5 

Z-Scores/T-Scores 

X 

X 

X 

X 

3 

Percentlle/Stanine 

X 

X 

X 

X 

2 

Frequency  Polygon 

X 

3 

Scattar  Diagram 

X 

X 

3 

Regression  Toward  the  Mean 

X 

1 

Standardised  Test/Interpretation 

X 

0 

Category  1: 
Category  2: 
Category  3: 
Category  4: 
Category  5: 


Content  Classification 
Purpose 

Behavioral  Objective 
Sample  Test  Item 
Resources 


X  Indicates  that  changes  made  In  the  category. 
Numbers  In  Category  5  Indicate  the  number  of 
resources  added  to  the  Instructional  module. 


97 


Cost  Analysis 

In  determining  the  personnel  costs  for  Phase  S—Revision  of 
Instructional  Modules  and  Tests,  only  time  in  excess  of  regular  class 
time  is  included  for  the  instructor.  The  time  estimates  for  the 
educational  technologist  do  not  Include  any  time  associated  with  the 
development  and  testing  of  the  ISS  programs.  From  an  analysis  of 
the  costs  associated  with  Phase  3  of  the  PIERIM  model  (see  Table  7), 
the  average  cost  of  revising  an  Instructional  module  and  the  related 
test  items  was  determined  to  be  $13.59.  The  cost  of  revising  the 
instructional  modules  and  tests  was  more  expensive  than  the  cost  of 
designing  the  materials  during  Phase  1  of  the  PIERIM  model.  The  cost 
of  producing  a  complete  set  of  Instructional  modules  and  tests, 
utilizing  the  ISS  programs  IMPROO  and  TEST,  was  determined  to  be  $1.65 
(i.e.,  $49.56  4  30)  per  set  of  materials  as  compared  to  the  cost  of 
$1.36  for  production  of  the  original  materials  manually.  The  original 
set  of  materials  contained  fewer  lines  of  type  and  it  is  reasonable  to 
expect  that  the  cost  of  manually  producing  the  revised  materials  would 
be  greater  but  there  is  not  a  method  to  accurately  project  the  cost 
estimate.  The  use  of  the  computer  to  produce  the  materials  has  the 
additional  advantages  of: 

1.  Producing  exactly  the  re(^i red  number  of  copies. 

2.  Automatically  sequencing  and  collating  the  materials. 

3.  Reducing  the  time  between  requesting  copies  of  the 
materials  and  receiving  the  completed  materials. 

4.  Simplifying  the  revision  and  modification  procedures 
over  the  traditional  manual  method  of  producing  the 
materials.  The  computer  pnxhiced  materials  have 
one  major  drawback  in  that  tite  paper  is  too  large 
for  the  learners  to  carry  in  conventional  notebooks. 


* 
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CHAPTER  VII 


IMPLEMENTATION  OF  THE  PIERIM  MODEL:  PHASE  4 

The  Instructional  modules  and  tests*  revised  and  produced 
during  Phase  3  of  the  PIERIM  model*  were  Implemented  In  the  evaluation 
unit  of  EEO  405— Classroom  Organization  and  Pupil  Evaluatlor  during 
the  Spring  quarter  of  the  1969-1970  school  year.  The  second  group 
of  learners  (I.e.*  Group  2)  to  participate  In  the  Implementation  of 
the  Instructional  modules  and  tests  were  elementary  education  majors. 
The  28  learners  were  In  either  their  junior  or  senior  year  and  enrolled 
In  additional  courses  which  placed  demands  upon  their  time. 

The  self  Instruction  environment  for  this  study  Is  opera¬ 
tionally  defined  by  the  following  characteristics  of  the  environment: 

1.  Each  learner  was  provided  w1^  a  set  of  the  Instruc¬ 
tional  mochiles. 

2.  The  principal  method  of  teaching  was  Individual  self  study. 

3.  There  were  no  scheduled  class  meetings. 

4.  The  audio  tape  and  text  resources  were  the  primary  resources. 

5.  The  resources  were  centrally  located  and  available  to 
the  learners  upon  request. 

The  major  activities  for  the  Implmaentatlon  the  revised  Instructional 
modules  were: 

1.  The  learners  completed  the  pretest  and  received  the 
Instructional  modules  for  the  self  Instruction  portion  of  the 
study  (sM  Appendix  0}* 

2.  The  learners  received  a  summery  of  their  pretest  perfbrmance 
(see  Figure  8)  and  the  iMtructor  discussed  the  Introduction 
to  the  set  of  Instructional  modules  with  the  learners. 
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3.  The  learners  interacted  with  the  instructional  modules  and 
resources  in  the  self-instruction  environment. 

4.  The  learners  completed  the  posttest  and  received  the  re¬ 
maining  instructional  modules  (see  Appendix  E)  which  were 
implemented  by  the  instructor  in  a  conventional  classroom 
environment. 

5.  The  learners  received  a  summary  of  their  posttest  per¬ 
formance  (see  Figure  9). 

6.  The  Instructor  completed  the  evaluation  unit  utilizing 
group  lecture  and  discussion  methods  to  implement  the 
remaining  instructional  modules. 

The  learners  were  expected  to  start  a  series  of  classroom 
observation  and  participation  activities  at  the  same  time  the  sat  of 
instructional  modules  were  implemented  in  the  self-instruction  environ¬ 
ment.  These  activities  sharply  reduced  the  amount  of  time  available 
to  the  learners. 

Phase  4  of  the  PIERIH  model  represents  the  first  implementation 
of  the  instructional  modules  in  a  self  instruction  environment  for 
which  the  instructional  modules  are  designed.  The  time  and  instruc¬ 
tional  modules  represent  fixed  variables  in  that  each  learner  received 
each  instructional  module  and  was  expected  to  complete  the  instruc¬ 
tional  modules  within  a  fixed  amount  of  time  between  the  pre-  and 
posttest. 

Because  the  stated  purpose  of  the  PIERIM  model  is  to  produce 
instructional  modules  for  use  in  a  self-instruction  environment,  the 
discussion  of  learner  performance  relates  to  the  set  of  16  instructional 
modules  implemented  in  the  self-instruction  environment.  The  remaining 
9  instructional  modules  used  the  instructor  both  as  the  primary 


10,3 


resource  and  the  evaluator  of  the  learners'  performance.  The  instructor 
reported  that  with  the  exception  of  one  learner,  the  learners'  performance 
met  his  standard  of  performance  for  the  set  of  9  instructional  modules. 

All  of  the  time  and  cost  estimates  are  based  on  the  design,  imple¬ 
mentation,  and  revision  of  the  entire  set  of  25  instructional  modules. 

Activities  Prescribed  by  PIERIM  Model 

The  activities  prescribed  by  the  PIERIM  model  for  Phase  4-- 
Implementation  and  Evaluation  of  Instructional  Modules  in  a  Self-Instruc¬ 
tion  Environment  are  identified  for  the  Instructor  (I)  and  the  Educa¬ 
tional  Technologist  (T)  as: 

19.  (I)  Administer  pretest. 

20.  (T)  Prepare  sutnmary  of  learner  performance  on  pretest. 

21.  (I)  Implement  instructional  modules  in  self-instruction 

environment. 

22.  (I)  Admin is'  "  posttest. 

23.  (T)  Prepare  summary  of  learner  performance  on  posttest. 

24.  (I)  Implement  instructional  modules,  evaluated  by  instructor, 

in  a  conventional  classroom  environment. 

25.  Repeat  activities  17  through  24  until  each  instructional 
module  achieves  the  standard  set  for  the  instructional  system. 


Chronology  of  Events 


The  activities  related  to  Phase  4”Implementatio.i  and  Evaluation 
of  Instructional  Modules  in  a  Self-Instruction  Environment  were  docu¬ 


mented  through  the  observation  of  the  activities  as  they  occurred.  Each 
activity  is  identified  as  to  date,  personnel  involved,  time  estimate  and 
a  description  of  the  activity. 


104 


DATE 

PERSONNEL 

TIME/HR. 

ACTIVITY 

4/20/70 

I 

1.0 

Administer  pretest 

4/20/70 

T 

.5 

Deliver  test  to  test  scoring 
service. 

4/20/70 

T 

1  .0 

Prepare  summaries  of  learner 
pretest  performance. 

4/21/70 

I 

1.0 

Discuss  the  Introduction 
to  the  set  of  instructional 
modules. 

4/29/70 

I 

1.0 

Administer  posttest. 

4/29/70 

T 

.5 

Deliver  test  to  test  scoring 
service. 

4/30/70 

T 

1.0 

Prepare  summaries  of  learner 
performance  on  the  posttest 

5/08/70 

I 

Evaluation  Unit  completed 

PIERIM  Model  Compared  with 
the  Actual  Activities 

The  activities  oi'iginally  specified  for  Phase  4— Implementation 
and  Evaluation  of  Instructional  Modules  in  a  Self-Instruction  Environ¬ 
ment  were  found  to  occur  in  the  sequence  specified  by  the  PIERIM  model. 
There  were  no  new  activities  identified  for  Phase  4. 

Summary  of  Learner  Performance 

The  learners'  performance  was  measured  for  the  set  of  16 
instructional  modules  using  the  same  form  of  a  42  item  multiple 
choice  test  as  the  pre-  and  posttest-  The  reliability  of  the 
pretest  (KR-20  *  .28)  was  lower  than  for  the  posttest  (KR-20  *  .71). 

The  means  and  standard  deviations  (see  Table  8)  are  reported  for 
each  of  the  16  instructional  modules  separately  and  for  the  total  test. 


TABLE  8.— Learner  Performance  data — Phase  4 
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Nine  of  the  inst^-uctional  modules  specified  that  the  learners' 
performance  would  be  evaluated  by  the  instructor.  The  instructor 
reported  that  with  the  exception  of  one  learner,  who  did  not  complete 
any  of  the  assignments  and  received  the  lowest  posttest  score,  the 
learners  satisfactorily  completed  the  evaluation  exercises  specified  by 
the  instructional  modules.  Twenty-four  of  the  27  learners  completed  all 
9  of  the  instructor  evaluated  assignments  and  the  remaining  3  learners 
completed  8  of  the  9  assignments. 

The  ISS  was  fully  operational  in  support  of  Phase  4  of  the  PIERIM 
model.  The  ISS  program  EVAL  produced  a  summary  of  the  learner's 
performance  on  both  the  pretest  and  posttest  (see  Figures  8  and  9) 
which  the  learner  received  the  day  after  the  test  was  completed.  The 
purpose  of  reporting  leat^ner  performance  by  instructional  module  is 
to  focus  the  learner's  attention  on  those  areas  in  which  his  performance 
is  less  than  the  standard  which  has  been  set  for  the  behavioral  objec¬ 
tive.  The  learners  indicated  that  the  sutnnary  of  their  pretest 
performance  was  used  to  guide  their  preparation  for  the  posttest. 

The  learners  did  not  indicate  that  they  wanted  any  additional  infor¬ 
mation  concerning  their  pretest  performance  other  than  a  review  of 
the  actual  test  items  missed.  The  EVAL  program  also  produced  a  sum¬ 
mary  report  for  the  instructor  (see  Figures  10  and  11)  of  the  pro- 
po« tion  of  the  learners  achieving  criterion  performance  for  each 
instructional  module. 

The  ISS  program  STAT  produces  a  report  which  sumnarizes  the 
pre-  and  posttest  performance  for  the  total  test  and  for  each  instruc¬ 
tional  module.  The  STAT  report  for  the  Reliability/Factors  Affecting 


Fig.  10. --Pretext  summary  report  producf'd  by  EVAL  program  -  group 
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module  (see  Figure  12)  is  representative  of  the  set  of  reports 
which  was  prepared  for  the  instructor.  The  STAT  reports  are  designed 
to  be  used  by  the  instructor  and  educational  technologist  in  con¬ 
nection  with  the  activities  related  to  Phase  3  of  the  PIERIM  model 
(i.e.,  revision  of  instructional  modules  and  tests).  The  format  of  the 
STAT  report  was  designed  to  present  information  which  had  been  re¬ 
quested  by  the  instructor  and  educational  technologist.  The  instructor 
reviewed  the  STAT  reports  and  stated  that  he  felt  overwhelmed  by  the 
magnitude  of  the  information  contained  in  the  reports.  Additional 
methods  of  summarizing  the  learners'  performance  for  the  set  of  instruc¬ 
tional  modules  were  investigated  and  are  reported  in  Chapter  VIII-- 
Instructional  Support  System. 

Learner  Participation 

The  activities  requested  of  the  learners  during  Phase  4  were 
related  to  the  evaluation  of  the  resources  and  a  questionnaire  con¬ 
cerning  their  attitude  toward  and  subjective  evaluation  of  the  instruc¬ 
tional  module,  tests,  and  procedures.  The  method  of  recording  their 
evaluation  of  the  resources  (see  Figure  13)  was  simplified  to  a  single 
page  and  the  learners  were  not  asked  to  make  a  written  statement 
concerning  their  decisions.  The  summary  of  the  learners'  responses 
(see  Table  9)  Indicates  that  the  textbook  (i.e.,  Lindvall)  and  the 
audio  tapes  were  consistently  the  most  popular  resources. 

Analysis  of  the  learners'  responses  to  the  questionnaire  (see 
Appendix  F)  for  the  learners  who  participated  In  Phase  4,  indicated 
that  not  all  learners  responded  to  each  question  and  one  learner 
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Numbers  indicate  the  nimtber  of  learners  who  rated  the  resources  as  useful. 
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(i.e.,  lowest  score  on  posttest)  did  not  respond  at  all.  The  learners' 
responses  to  the  questions  indicate: 

1.  Nineteen  of  27  learners  thought  that  they  were  provided  enough 
information  to  evaluate  their  performance  on  the  pre-  and  posttest. 

The  learners  who  did  not  think  the  information  was  adequate  wanted  to 
review  the  test  items  and  answers  for  the  tests. 

2.  Twenty-two  of  26  learners  indicated  that  their  pretest  performance 
was  used  to  guide  their  preparation  for  the  posttest.  Four  learners 
indicated  that  they  did  not  use  the  information  for  any  purpose. 

3.  Thirteen  of  27  learners  reported  that  they  read  the  entire  set 
of  instructional  modules  in  preparation  for  the  posttest  and  eight 
learners  stated  that  they  only  read  specific  instructional  modules. 

Seven  learners  reported  that  they  followed  the  same  procedure  of  reading 
the  instructional  module,  reading  the  text,  and  listening  to  the  audio 
tape  for  each  instructional  module. 

4.  Twenty-four  of  27  learners  indicated  that  the  sample  test  items 
adequately  described  the  type  test  items  on  the  posttest. 

5.  Twelve  of  24  learners  indicated  that  they  had  no  difficulty  in 
using  the  library  books  and  12  indicated  that  they  did  not  use  the 
books  or  the  books  were  not  available. 

6.  Nineteen  of  25  learners  reported  no  difficulty  in  using  the 
audio  tapes  and  6  reported  that  the  tapes  were  either  not  available  or 
complained  of  the  poor  quality  of  the  reproduction.  Only  one  learner 
indicated  that  the  tapes  were  generally  vague  and  confusing. 

7.  The  advantages  of  using  instructional  modules  reported  by  the 
learners  included  the  following:  the  instructional  modules  were 
specific,  the  learner  could  work  at  his  own  pace,  material  already 
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mastered  could  be  omitted,  there  was  more  than  one  resource  for  use  and 
a  learner  could  repeat  a  unit  until  he  was  confident  he  had  mastered 
the  instructional  module. 

8.  The  disadvantages  of  using  instructional  modules  and  the  self- 
instruction  procedures  were  reported  to  include  the  following:  the 
instructional  modules  were  too  narrow  and  detailed,  the  learner  could 
not  ask  questions,  the  instructional  modules  lacked  motivation  for  the 
learner  to  master  the  material,  and  application  of  the  stated  behaviors 
was  not  apparent. 

9.  Eighteen  of  26  learners  reported  that  they  would  take  a  course 
with  instructional  modules  in  preference  to  the  same  course  without 
instructional  modules. 

The  proportion  of  "'earners  in  Phase  4  who  would  select  the  course  with 
instructional  modules  was  only  slightly  higher  than  the  proportion  from 
Phase  2  (i .e.,  12  of  18). 

Comparison  of  Phase  2  and  Phase  4 

A  comparison  of  the  similarities  and  differences  which  existed 
during  Phase  2— Implementation  and  Evaluation  of  Instructional  Module 
in  a  Conventional  Classroom  Environment  and  Phase  4— Implementation  and 
Evaluation  of  Instructional  Modules  in  a  Self-Instruction  Environment 
provides  a  frame  of  reference  for  the  analysis  and  interpretation  of  the 
learner  performance  data  reported  in  Tables  4  and  8.  The  pre-  and 
posttest  means  for  the  two  groups  of  learners  are  reported  in  Table  10. 
The  similarities  which  existed  between  the  two  implementations  of  the 
instructional  modules  included: 
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1.  Course--The  evaluation  unit  of  EED  405--Classroom  Organization  and 
Pupil  Evaluation  was  used  to  implement  the  instructional  modules. 

2.  Inst'^uctor--The  same  graduate  assistant  instructor  was 
given  complete  responsibility  for  the  evaluation  unit. 

3.  Population--The  learners  were  all  elementary  education 
majors  in  either  their  junior  or  senior  year  at  The  Florida  State 
University. 

4.  Length  of  Unit--The  evaluation  unit  was  allocated  a 
total  of  nine  one-hour  class  sessions. 

5.  Concurrent  Activities—The  learner's  were  all  enrolled 

in  additional  courses  which  placed  demands  on  their  time.  The  demands 
placed  on  the  learners'  time  by  activities  not  directly  associated 
with  the  evaluation  unit  are  assumed  to  be  equivalent  for  both  of  the 
implementations  of  the  instructional  modules. 

6.  Attendance--The  learners  were  not  required  to  attend  class 
du'^ing  the  implementation  of  the  instructional  modules.  They  were 
required  to  complete  the  pre-  and  posttest  for  the  unit. 

The  significant  differences  between  the  two  implementations 
of  the  instructional  modules  are: 

1.  Instructor— The  instructor  assisted  with  the  entire  course 
during  Phase  2  and  only  with  the  evaluation  unit  during  Phase  4. 

2.  Instructional  Modules— The  revision  of  the  instructional 
modules  resulted  in  substantially  different  instructional  materials 
being  used  in  the  two  phases  of  the  model.  The  magnitude  of  the  revisions 
for  each  instructional  module  is  summarized  (see  Table  6)  and  depicted 
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in  the  set  of  materials  related  to  the  instructional  module,  Reliability/ 
Factors  Affecting  (see  Appendix  B). 

3. '  Test  Items— A  set  of  42  multiple  choice  test  items  was 

used  to  measure  the  learners'  performance  on  the  16  instructional  modules 
which  specified  multiple  choice  items  as  the  method  of  evaluation. 

There  were  3  test  items  replaced  and  11  test  items  modified  during  the 
revision  of  the  instructional  materials. 

4.  Testing  Procedures --The  time  between  the  pre-  and  posttest 
was  reduced  from  16  calendar  days  during  Phase  2  to  8  calendar  days 
during  Phase  4. 

5.  Primary  Resource— The  instructor  and  text  served  as  the 
primary  resources  during  Phase  2  and  the  resources  identified  for  each 
instructional  module  were  the  primary  resources  during  Phase  4. 

6.  Availability  of  Additional  Resources— The  environment 

for  Phase  2  was  a  conventional  classroom.  The  self-instruction  environ¬ 
ment  of  Phase  4  was  the  Florida  State  University  library.  All  of  the 
resources  were  available  in  one  building  and  additional  resources  could 
be  easily  located,  if  needed. 

7.  Availability  of  Instructor— The  instructor,  during  Phase 

2,  was  the  primary  resource  and  was  available  during  the  class  period 
and  by  appointment.  During  Phase  4,  the  instructor  was  available  in 
his  office  during  the  regular  class  period  and  by  appointment. 

8.  Instructional  Materials— The  entire  set  of  instructional 
modules  was  given  to  the  learners  at  the  beginning  of  Phase  2  and  the 
instructional  modules  were  divided  into  two  sets  of  materials  during 
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Phase  4  (see  Appendix  D  and  E).  The  set  of  16  instructional  modules 
which  are  evaluated  by  multiple  choice  test  items  was  given  to  the 
learners  after  the  pretest  and  the  remaining  instructicnal  modules  were 
given  to  the  learners  after  the  posttest. 

9.  Class  Size— Nineteen  learners  participated  in  Phase  2  and 
28  learners  participated  in  Phase  4  of  the  PIERIM  model. 

Interpretation  of  Learner  Performance 

The  learners'  performance  can  be  expected  to  deviate  from 
the  performance  predicted  by  criterion-referenced  measurement  and 
mastery  models  of  learning  to  the  extent  that  the  following  assumpt1oi)St 
implicit  in  the  procedures  used  to  design  and/or  implement  the 
instructional  modules  and  tests,  are  violated: 

1.  Learners  enter  the  instructional  system  in  an  unlearned 
state. 

2.  Learners,  who  interact  with  the  instructional  resources 
specified,  change  from  an  unlearned  to  learned  state. 

3.  Learners  possess  any  prerequisite  competencies  required 
to  interact  with  the  instructional  resources  that  are 
identified  for  the  instructional  modules. 

4.  Learners  have  sufficient  time  to  achieve  mastery  on  each 
instructional  module. 

5.  Test  items,  for  each  instructional  module,  represented  a 
homogeneous  sample  of  the  performance  described  by  the 
behavioral  objective. 

The  learners'  performance  was  measured  for  the  set  of  16 
instructional  modules  using  the  same  form  of  a  42  item  multiple  choice 
test  as  both  the  pre-  and  posttest  in  a  One  Group  Pretest-Posttest 
Design.  Revisions  were  made  to  the  test  during  Phase  3  and  this  factor 
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should  be  considered  when  comparing  the  performance  of  Group  1  (i.e., 
Conventional  Classroom  Group)  and  Group  2  (i  e.,  Self-Instruction  Group). 
The  sample  size  for  Grouo  1  and  Group  2  were  19  and  28  learners  respec¬ 
tively. 

Violation  of  Statistical  Assumptions 

The  interpretation  of  learner  performance  data  is  further 
complicated  by  the  use  of  intact  classroom  groups  to  study  the 
effects  of  the  instructional  materials  and/or  procedures  on  the 
learners'  performance.  The  use  of  intact  classroom  groups  violates  one 
of  the  basic  underlying  assumptions  of  inferential  statistics  (i.e., 
random  sampling  of  learners  from  the  population).  The  assumption  that 
the  underlying  distribution  of  the  trait  being  evaluated  approximates 
the  normal  distribution  is  violated  as  the  actual  effectiveness  of  the 
instructional  materials  and/or  procedures  approach  their  theoretical 
limit  of  100  percent  effectiveness.  Non-parametric  statistics  were 
selected  for  analysis  of  the  learner  performance  data.  Non-porametric 
statistics  (i  e.,  phi  coefficients  and  McNemar's  Test)  were  selected 
to  be  reported  by  the  ISS  program  STAT  (see  Figure  12)  because  there 
are  no  assumptions  required  concerning  the  underlying  distribution 
of  the  performance  data. 

The  purpose  of  designing  and  inplementing  the  instructional 
modules  in  a  self-instruction  environment  was  for  the  learners  to 
achieve  at  least  the  standard  of  performance  specified  for  each  of 
the  instructional  modules.  Learning  is  inferred  from  gains  in  the 
proportion  of  learners  achieving  the  standard  of  performance  from 
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pi^etest  to  posttest.  It  is  important  to  remember  that  the  research 
design  utilized  (i.e.,  One  Group  Pretest-Posttest  Design)  makes  it 
impossible  to  separate  the  gains  attributable  to  the  effects  of  testing 
from  the  gains  attributable  to  the  instructional  treatment.  Utilizing 
the  proportion  of  learners  achieving  at  least  the  standard  of  per¬ 
formance  on  the  pretest  (see  Figure  10)  and  posttest  (see  Figure  11) 
the  gains  from  pretest  to  posttest  and  the  ratio  of  the  gains  to  poten¬ 
tial  gain  are  reported  for  each  instructional  module  (see  Table  11). 

Any  arbitrary  standard  can  be  selected  as  the  performance  stan¬ 
dard  for  a  system  model  for  design  of  instruction.  For  purposes  of 
illustrating  the  use  of  a  standard  of  performance  for  a  system  model 
for  design  of  instruction.  70  percent  is  selected  as  the  system  stan¬ 
dard  for  the  PIERIH  model.  The  learners  achieved  the  system  stan¬ 
dard  of  performance  on  four  of  the  16  instructional  modules  on  the 
pretest  and  for  10  of  the  16  instructional  modules  on  the  posttest 
(see  Table  11).  There  vvould  be  reason  to  suspect  that  for  at  least  the 
four  instructional  modules  on  which  the  system  standard  of  70  percent 
was  achieved  on  the  pteUst  that  the  topic  had  been  previously  taught 
in  other  education  courses  or  the  instructional  objective  was  so 
obvious  as  not  tc  require  instruction.  A  comparison  of  the  ratios  of 
gains  to  potential  gains  requires  the  assumption  that  a  gain  from  .800 
to  .900  (i.e..  .10/. 20  •  .500)  is  equivalent  to  a  gain  of  from  .400  to 
.700  (i.e.,  30/. 60  -  .500). 

Role  of  the  Learner 

The  learners  were  given  three  distinct  roles  within  the 
implementation  of  the  instructional  modules.  The  learners  interacted  with 
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the  instructional  modules,  recommended  changes  in  the  instructional 
nwdules  and/or  procedures,  and  expressed  their  attitude  toward  the  instruc¬ 
tional  modules  and/or  procedures.  The  learners'  interaction  with  the 
instructional  modules  was  not  closely  monitored  during  either  Phase  2  or 
Phase  4  of  the  PIERIM  model.  Class  attendance  during  either  Phase  2 
was  optional  and  reported  by  the  instructor  to  average  from  10  to  13 
of  the  19  learners  per  day.  Using  self  report  techniques,  the  median 
number  of  days  prior  to  the  posttest  on  which  the  learners  started 
their  preparation  for  the  posttest  was  five  days  for  Phase  4.  The 
learners  reported  that  they  devoted  from  4  to  14  hours  {X  =  6.9  hours) 
preparing  for  the  posttest. 

The  learners  were  asked  in  the  introduction  to  the  set  of  instruc¬ 
tional  modules,  during  Phase  2,  to  assume  a  professional  responsibility 
and  assist  the  instructor  by  the  following: 

1.  Evaluate  the  components  of  each  instructional  module  by 
specifically  attending  to  the  clarity  of  the  stated  behavioral  objective 
and  the  congruence  (i.e.,  agreement)  between  the  behavioral  objective 
and  test  situation. 

2.  State  whether  the  resource(s)  was  adequate  for  the  learner 

to  achieve  the  stated  behavioral  objective.  If  multiple  resources  or 
chapters  within  a  single  resource  were  cited,  the  learner  was  to  indicate 
which  of  the  resources  was  the  most  helpful  in  achieving  the  stated 
behavioral  objective. 

3.  Identify  additional  resources  that  could  be  used  by  a  learner 
to  master  the  stated  behavioral  objective. 
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4.  Propose  other  evaluation  topics,  concepts  or  activities  for  which 
instructional  modules  should  be  prepared. 

5.  Identify  any  words  or  topics  which  were  either  confusing  to 
learners  or  required  more  than  an  average  amount  of  study  for  mastery. 

6.  Keep  a  diary  of  events  in  which  tne  learner  indicated  any  strong 
positive  or  negative  feeling  toward  the  procedures  being  used,  materials 
provided,  resources  identified,  sequence  of  topics,  etc.  Of  the  six 
proposed  activities,  the  only  activities  which  the  learners  completed 
and  which  proved  useful  during  Phase  3  were  activities  one  and  five. 

There  were  too  many  activities  expected  of  the  learners  and  they  either 
completely  failed  to  complete  the  remaining  activities  or  their  descrip¬ 
tions  were  too  general  to  be  of  any  help  in  the  revision  of  the  instruc¬ 
tional  modules. 

The  activities  requested  of  the  learners  during  Phase  4  were 
subsequently  reduced  to  an  evaluation  of  the  resources  and  a  questionnaire 
concerning  their  attitude  toward  and  subjective  evaluation  of  the  instruc¬ 
tional  modules  and  procedures.  The  method  of  evaluating  the  resources 
was  simplified  to  a  single  page  and  the  learners  were  not  asked  to  make 
any  written  statements  concerning  their  decisions. 

Cost  Analysis 

In  determining  the  personnel  costs  for  the  implementation  of  the 
instructional  modules  and  tests  only  time  in  excess  of  regular  class  time 
is  reported  for  the  instructor.  The  instructor  was  not  engaged  in  any 
activities  other  than  the  normal  teaching  activities  during  the  imple¬ 
mentation  of  the  instructional  modules.  The  only  costs  involved  in  the 
implementation  of  the  instructional  modules  in  a  self-instruction 


environment  are  related  to  the  production  of  reports  of  learner  performance. 
The  average  cost  of  the  summary  report  produced  for  the  learner  by  the  ISS 
program  EVAL  was  six  cents  per  report.  The  summary  report  of  the  pre- 
and  posttest  performance  produced  by  the  ISS  program  STAT  was  $4.00  for 
the  report.  A  summary  of  the  costs  related  to  Phase  4  is  presented  in 
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INSTRUCTIONAL  SUPPORT  SYSTEM 

The  Instructional  Support  System  (ISS)  is  defined  by  the  set 
of  capabilities  which  are  required  to  support  the  implementation 
of  the  PIERIM  model.  The  implementation  of  the  ISS  in  support 
of  Phase  4  of  the  PIERIM  model  is  reported  in  relation  to:  (1)  test 
scoring  services,  (2)  computer  programs,  (3)  resource  center,  (4)  methods 
used  to  establish  the  transferability  of  the  instructional  modules, 
and  (5)  efforts  to  develop  a  Revision  Indicator  for  use  in  relation 
with  the  STAT  program. 

Test  Scoring  Service 

The  Division  of  Instructional  Research  and  Service  at  The 
Florida  State  University  offers  a  test  scoring  and  analysis  service 
for  use  by  the  instructional  staff.  The  characteristics  required 
for  the  evaluation  of  a  learner's  performance  include  methods  for: 

(1)  presenting  the  test  items,  (2)  recording  the  learner's  responses, 

(3)  evaluating  the  learner's  responses,  (4)  summarizing  the  learner's 
responses,  and  (5)  producing  a  record  of  the  learner's  responses.  The 
test  scoring  service  provided  the  means  of  accomplishing  all  of  the 
requirements  with  the  exception  of  presenting  the  test  items.  The  test 
service  programs  provided  both  total  test  and  item  analysis  for  the 
total  group  of  learners.  The  response  record  of  each  learner  was 


punched  in  a  format  which  was  interpretable  by  a  computer  program 
written  in  FORTRAN  IV  programming  language. 

Computer  Programs 

The  documentation  of  each  of  the  four  ISS  computer  pro- 
grams--IMPROD,  TEST,  EVAL,  and  STAT  -is  reported  in  Appendix  G. 

The  computer  programs  serve  three  functions:  (1)  Production  of 
instructional  modules  and  tests,  (2)  evaluation  of  learner  per¬ 
formance  ,  and  (3)  analysis  and  summary  reports  of  learner  per¬ 
formance  data.  The  specific  capabilities  required  of  the  ISS  pro¬ 
grams  for  the  production  of  instructional  modules  and  tests  include: 
(1)  production  of  any  specified  number  of  sets  of  instructional 
modules  and  tests,  (2)  sequencing  of  the  Instructional  modules 
and  test  items  In  any  order  requested,  and  (3)  revision  of  instruc¬ 
tional  modules  and  tests  possible  at  the  lowest  practical  level. 

The  ISS  programs,  IMPROD  and  TEST,  possess  each  of  the  capabilities 
specified  with  the  line  of  print  being  used  as  the  basic  unit  for 
revision  of  the  instructional  modules  and  test  items.  The  advan¬ 
tages  associated  with  the  ISS  computer  programs  are  related  to  the 
transferability  of  the  programs  among  universities  and  ease  with 
which  they  can  be  programmed  and  revised.  It  is  possible  that  minor 
modifications  to  the  programs  would  be  required  before  they  would 
operate  on  computer  systems  other  than  a  Control  Jata  Corporation  6400 
series  computer. 

The  basic  unit  produced  through  the  implementation  of  the 
PIERIM  model  was  the  instructional  module,  and  learner  performance  was 
analysed  and  sunmarized  by  the  ISS  program  EVAL  by  instructional  modul 


The  characteristics  required  for  the  analysis  and  summary  of  learner 
performance  include:  (1)  analysis  and  summary  of  learner  performance 
by  instructional  module,  (2)  comparison  of  the  learner's  performance 
with  the  standard  of  performance  established  for  the  instructional 
module,  and  (3)  summary  of  the  group'^>  performance  for  each  instruc¬ 
tional  module.  The  ISS  program  EVAL  was  designed  for  the  analysis  of 
a  single  test  and  is  the  most  difficult  ISS  program  to  modify.  The 
entire  EVAL  program  needs  to  be  redesigned  so  that  the  relationship 
between  test  items  and  instructional  modules  can  be  read  into  the 
program  at  the  time  the  data  is  analyzed. 

The  analysis  and  summary  of  the  pre-  and  posttest  learner 
performance  data  by  the  ISS  program  STAT  was  designed  on  two  principles: 
(I)  all  of  the  pre-  and  posttest  data  and  computed  statistics  are 
available  at  the  end  of  the  program  run,  and  (2)  the  instructor  specifies 
the  information  and  format  of  the  summary  reports  of  learner  performance. 
The  STAT  program  has  a  major  disadvantage  of  requiring  a  minimum  of 
76,000  units  of  computer  core  to  operate.  It  is  possible  that  through 
the  use  of  more  sophisticated  programming  techniques  that  the  STAT 
program  could  be  made  operational  on  a  smaller  computer  system. 

The  cost  of  p^'oducing  the  ISS  computer  programs  cannot  be 
precisely  determined.  The  educational  technologist  prograirmed,  key 
punched,  and  tested  each  of  the  programs.  The  best  estimate  of  the 
time  required  Is  80  hours  or  one  man  month  for  a  graduate  assistant. 

The  computer  costs  are  estimated  on  the  basis  of  4  trial  runs  with 
program  listings  for  each  of  the  programs.  The  estimated  cost  of 
$434  Includes:  (1)  $320  personnel— 80  hours  at  $4  pe**  hour,  (2)  $7 
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computer  cards--7000  cards  at  $1  per  1000,  (3)  $37  computer  time-- 
300  seconds  at  $.042  per  second  and  200  seconds  at  $.062  per  second, 

(4)  $70  computer  printing--50,000  lines  of  print  at  $.0014  per  line. 

The  cost  of  producing  sets  of  the  instructional  modules  and 
tests  are  based  on  production  runs  of  30  sets  of  each  of  the  materials. 
The  production  runs  utilized  binary  source  decks,  and  program  listings 
were  not  produced.  Tne  cost  of  the  summary  reports  of  learner  per¬ 
formance  are  based  on  the  actual  production  costs  associated  with  the 
implementation  of  the  programs  during  Phase  4  of  the  PIERIM  model. 

The  total  cost  of  producing  the  instructional  modules,  tests,  pre-  and 
posttest  reports  of  learner  performance  for  the  learner,  and  pre- 
and  posttest  report  of  learner  performance  for  the  instructor  was 
determined  to  be  $56.93.  If  the  18%  overhead  cost  is  added  to  the 
computer  costs,  then  the  total  cost  is  $67.17.  The  average  cost  of 
supporting  the  impleinentation  of  an  instructional  module  was  $2.28  or 
$2.69  with  the  overhead  costs. 


Resource  Center 

The  services  provided  by  the  university  library  included  place¬ 
ment  of  the  books  identified  as  resources  for  the  instructional  modules 
in  the  reserve  room  of  the  library,  storage  of  the  audio  tapes,  and 
facilities  for  listening  to  the  tapes.  The  library  also  provided  the 
instructor  with  a  monthly  summary  of  the  tape  usage.  The  usage  report 
was  for  each  audio  tape  and  reported  the  nun^er  of  times  the  tape  was 
used  and  the  time  of  day  used.  The  tapes  appear  to  have  a  great  poten¬ 
tial  for  increasing  the  learners'  involvement  with  the  Instructional 
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resources.  The  audio  tapes  were  produced  as  summaries  for  the  instructional 
modules  and  the  learners  generally  found  them  to  be  a  useful  resource 
in  preparing  for  the  posttest  (see  Table  9). 

Transferability  of  Instructional  Modules 

The  utility  of  the  PIERIM  model  and  the  ISS  was  determined 
to  be  dependent  upon  two  major  factors.  First,  the  ability  of 
other  faculty  members  to  agree  on  the  relative  importance  of  a 
teacher  candidate's  being  able  to  demonstrate  the  behaviors  described 
under  the  subheading  PURPOSE  of  the  instructional  modules  The 
second  factor  relates  to  the  time  required  to  modify  the  instructional 
modules  for  use  by  another  faculty  member.  Expert  opinion  was  used 
to  investigate  the  first  factor  and  a  replication  of  the  one  week 
self  study  was  used  to  investigate  the  second  factor. 

Ten  faculty  members  were  selected  to  review  and  evaluate  the 
relative  importance  of  the  set  of  instructional  modules  prepared  for 
use  in  the  conventional  classroom  environment-  The  faculty  members 
were  selected  on  the  basis  of  their  having  either  taught  the  course 
for  which  the  instructional  modules  were  prepared  or  courses  related 
to  the  area  of  tests  and  measurement  offered  by  the  Department  of 
Educational  Research.  The  cover  letter  and  a  sample  from  the  set  of 
instructional  modules  (see  Appendix  H)  are  representative  of  the 
materials  sent  to  the  faculty  members,  six  from  the  Department  of 
Elementary  Education  and  four  from  the  Department  of  Educational  Research. 
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The  faculty  members  were  asked  to  first  select  those  Instruc¬ 
tional  modules  which  described  behaviors  they  did  not  feel  a  teacher 
candidate  should  be  expected  to  master.  They  were  then  asked  to 
equally  divide  the  remaining  instructional  modules  into  4  categories: 

(1)  most  important.  (2)  above  average  in  importance,  (3)  below  average 
in  importance,  and  (4)  least  important.  Values  of  4,  3,  2,  and  1 
were  assigned  to  the  corresponding  categories  and  the  responses  were 
summarized  for  each  instructional  module.  The  responses  are  reported 
for  the  Department  of  Elementary  Education  and  Department  of 
Educational  Research  separately  and  for  the  total  group  (see  Table  13). 
Two  factors  could  bias  the  results:  (1)  two  faculty  members  from  the 
Department  of  Elementary  Education  did  not  return  the  materials,  and 

(2)  one  of  the  faculty  members  did  not  have  an  equal  number  of  instruc¬ 
tional  modules  in  each  of  the  4  categories.  It  was  inferred  from 
conversations  with  the  faculty  members  who  did  not  return  the  Instruc¬ 
tional  modules  that  they  perceived  a  conflict  between  their  preferred 
teaching  style  and  the  degree  of  structure  represented  in  the  instruc¬ 
tional  modules. 

A  Spearman  Rank  Correlation  Coefficient  (r^),  corrected  for 
tied  ranks  (Siegel,  19S6)  was  calculated  to  determine  If  associations 
existed  between  the  ranking  of  the  Instructional  modules  by  the  faculty 
members  from  the  Department  of  Elementery  Education  and  by  the  faculty 
members  from  the  Department  of  Educational  Research.  The  null  hypo¬ 
thesis  that  the  two  rankings  were  unrelated  was  rejected  (alpha  «  .01} 
for  the  entire  set  of  Instructional  modules  (r^  •  .71).  When  the  16 
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TABLE  13. --Evaluation  of  Instructional  Modules- -Expert  Opinion 


Instructional  Module 

A 

B 

C 

Pretest/Posttest 

13 

9 

22 

Behavioral  Objectives 

16 

15 

31 

Test  Items 

16 

12 

28 

Percentile  Ranks 

9 

11 

20 

Measures  of  Central  Tendency 

11 

7 

18 

Normal  Distribution 

7 

8 

15 

Normal  Curve 

10 

7 

17 

Correlation  Coefficient 

3 

G 

9 

Correlation/Scatter  Diagram 

5 

5 

10 

Validity 

5 

7 

12 

Reliabllity/Factors  Affecting 

8 

11 

19 

Rel 1 abl 1 i ty/ Interpretation 

7 

10 

17 

Standard  Error  of  Measurement 

8 

10 

18 

Types  of  Test 

11 

6 

17 

Test  Norms/ Intel  1 i gence  Quotient 

15 

12 

27 

Standardized  Test  Information 

11 

9 

20 

Test  Items/Behavioral  Objectives 

15 

16 

31 

Percentile  Ranks 

1 

8 

9 

Measures  of  Central  Tendency 

7 

10 

17 

Z-Scores/T-Scores 

4 

5 

9 

Percent! le/Stanine 

7 

6 

13 

Frequency  Polygon 

2 

4 

6 

Scatter  Diagram 

10 

9 

19 

Regression  Toward  the  Mean 

4 

6 

10 

Standardized  Test/ Interpretation 

13 

13 

26 

Numbers  represent  the  sum  of  the  faculty  member  ratings. 


A  •  Department  of  Elementary  Education 
B  >  Department  of  Educational  Research 
C  >  Total  Group 

Total  Set  of  Instructional  Modules  r^(AB)  ■  .71 

Set  of  First  16  Instructional  Modules  r  (AB)  ■  .SB  ** 

s 

Set  of  last  9  Instructional  Modules  r^CAB)  <*  .78  ** 
••Significant  at  .01  level 
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instructional  modules  (r^  _  .58)  evaluated  by  objective  test  items,  and 
the  9  instructional  modules  (r^  =  .78)  evaluated  by  the  instructor, 
were  analyzed  separately;  the  null  hypothesis  was  likewise  rejected. 

The  faculty  members'  responses  to  the  question:  Can  this 
behavior  be  adequately  evaluated  using  the  stated  behavioral  objective 
and  test  situation?  are  much  more  difficult  to  interpret.  The 
response  alternatives  available  to  the  faculty  member  were:  yes,  uncertain, 
and  no.  One  of  the  faculty  members  from  the  Department  of  Elementary 
Education  responded  "no"  for  each  of  the  instructional  modules.  The 
summary  of  the  faculty  members'  responses  (see  Table  14)  indicates 
that  the  perceived  agreement  between  the  test  situation  and  the  behavioral 
objective  which  it  is  intended  to  evaluate  is  higher  when  the  instructor 
evaluated  the  learner's  performance  (Y  *  5.7  for  yes  category)  than 
when  objective  test  items  were  used  to  evaluate  the  learner's  performance 
(X  =  4.8  for  yes  category). 

The  faculty  members  estimated  the  percentage  of  the  evaluation 
unit  that  the  set  of  instructional  modules  would  represent  if  they  were 
to  teach  the  course.  The  Department  of  Elementary  Education  faculty 
members  reported  percentages  of  0%,  50%,  70%,  and  100%  in  contrast  to 
a  narrower  range  of  values  reported  by  faculty  members  from  the 
Department  of  Educational  Research  of  50%,  60%,  70%  and  80%. 

The  evidence  indicates  that  the  instructional  modules  produced 
through  the  Implementation  of  the  PIERIM  model  describe  a  set  of  learner 
competencies  for  which  moderate  agreement  was  obtained  from  eight 
faculty  members  concerning  the  relative  importance  of  the  competencies 
described.  There  was  generally  less  agreement  concerning  the  adequacy 
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TABLE  14. --Adequacy  of  Test  Situation--Expert  Opinion 


Frequency  of  Response 

instructional  modules 

Tes 

uncertain 

No 

Protest/Posttest 

4 

2 

2 

Behavioral  Objectives 

3 

3 

2 

Test  Items 

4 

2 

2 

Percentile  Ranks 

6 

1 

1 

Measures  of  Central  Tendency 

4 

2 

2 

Normal  Distribution 

4 

2 

2 

Normal  Curve 

7 

0 

1 

Correlation  Coefficient 

4 

2 

2 

Correlation/Scatter  Diagram 

7 

0 

1 

Validity 

3 

4 

1 

Reliability/Factors  Affecting 

5 

1 

2 

Rel i abi 1 i tV/ Interpretation 

6 

1 

1 

Standard  ^rror  of  Measurement 

3 

2 

3 

Types  of  Tests 

4 

2 

2 

Test  Norms/  Intelligence  Quotient 

5 

2 

1 

Standardized  Test  Information 

6 

1 

1 

Test  Items/feehavioral  Objectives 

6 

0 

2 

Percentile  Ranks 

6 

1 

1 

Measures  of  Central  Tendency 

7 

0 

1 

Z-Scores/T-Scores 

4 

0 

4 

Percent! le/Stanine 

6 

1 

1 

Frequency  Polygon 

5 

2 

1 

Scatter  Diagram 

6 

1 

1 

Regression  Toward  the  Mean 

5 

2 

1 

Standardized  Test/Interpretation 

6 

1 

1 

Numbers  »  frequency  of  responses  to  the  question;  "Can  this 
behavior  be  adequately  evaluated  using  the  stated 
behavioral  objective  and  test  ^situation?" 
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of  the  objective  test  items  for  the  evaluation  of  the  learners' 
attainment  of  the  stated  behavioral  objectives-  For  most  of  the 
faculty  menters,  the  set  of  instructional  modules  would  represent 
a  significant  portion  of  the  course  if  they  were  to  teach  the 
evaluation  unit. 

The  replication  of  the  one  week  self  study  was  used  to 
determine  the  tine  required  to  modify  the  instructional  modules, 

Tne  ISS  was  designed  using  programming  techniques  which  were 
intended  to  minimize  the  time  required  to  w»dify  the  instructional 
modules.  The  set  of  instructional  moAiles  was  reviewed  by  an 
instructor  of  an  undergraduate  educational  psycliology  course  and 
the  first  16  instructional  modules  were  selected  to  be  included  in 
a  one  week  unit,  "Performance  Assessment,"  for  the  course.  The 
instructor  requested  that  the  text  being  used  in  the  course  be 
Included  in  the  resources  for  the  instructional  modules  and  that 
the  text  used  in  the  original  course  be  deleted.  The  activities 
required  to  modify  the  instructional  modules  were  identified  as; 

1.  Review  the  set  of  instructional  modules  (4.0  hrs.). 

2.  Select  appropriate  instructional  modules  (.5  hrs.). 

3.  Specify  resources  to  be  added  or  deleted  (2.0  hrs.). 

4.  Key  punch  change  cards  for  instructional  modules  (1.0  hrs.). 

5.  Produce  the  required  number  of  sets  of  instructional  modules. 
(3.0  hrs.).  The  total  time  required  to  complete  the  entire  process 
was  seven  calendar  days.  The  changes  in  the  set  of  instructional 
modules  included  the  following: 

1.  Introduction--changed  the  name  of  the  unit,  course  and  date 
of  the  posttest. 
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2.  One  additional  resource  added  to  each  of  3  instructional 
modules  and  the  bibliography, 

3.  One  resource  deleted  from  each  of  15  instructional  modules 
and  the  bibliography,  and 

4.  Evaluation  of  Resources  was  changed  to  include  or  delete  the 
appropriate  resources. 

Revision  Indicator  for  Instructional  Modules 
When  the  instructor  of  the  elementary  education  course  reviewed 
the  set  of  Summary  reports  produced  by  the  ISS  program  STAT,  he  reported 
that  the  volume  of  information  contained  in  the  reports  was  overwhelming, 

A  single  rank  indicator  for  each  instructional  module  would  be  an  asset 
to  the  instructor  and  educational  technologist  by  directing  their  efforts 
during  the  revision  of  the  instructional  modules.  Neither  the  summary 
reports  produced  by  the  program  STAT  or  the  Revision  Indicator  have 
actually  been  utilized  to  support  Phase  3  of  the  PIERIM  model. 

The  rationale  for  the  Revision  Indicator  was  to  select  a  number 
of  statistics,  which  were  available  to  the  instructor  and  educational 
technologist,  and  predict  the  direction  in  which  each  statistic  would 
be  expected  to  change  on  the  basis  of  criterion-referenced  measurement 
and/or  mastery  models  of  learning.  The  Revision  Indicator  is  a  single 
composite  value  derived  from  the  following  statistics: 

1.  Mean— The  posttest  mean  is  predicted  to  be  greater  than  the  pretest 
mean.  The  means  for  Group  1  and  Group  2  (see  Table  10)  indicate  that 

the  mean  of  each  instructional  module  did  in  fact  increase  from  pretest 
to  posttest. 

2.  Standard  Deviation— The  posttest  standard  deviation  is  predicted 
to  be  less  than  the  pretest  standard  deviation.  The  standard  deviations 
for  Group  1  (see  Table  4)  and  Group  2  (see  Table  8)  indicate  that  for 
.some  of  the  instructional  modules  the  standard  deviations  changed  in 

the  opposite  direction. 
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3.  Maximum  Pretest  Score--Learners  who  achieve  a  maximum 

score  on  the  pretest  ai'e  p<-ed1cted  to  achieve  mastery  on  the  posttest. 

4.  Posttest  Scores  of  Zero— Less  than  5%  of  the  learners  are 
predicted  to  be  in  an  unlearned  state  on  each  of  the  items  related  to 
an  instructional  module 

5.  Phi  Coefficients--Each  of  the  inter  item  phi  coefficients 
for  a  set  of  items  related  to  an  instructional  module  are  predicted 
to  be  positive.  The  total  number  of  negative  phi  coefficients  is 
calculated  for  the  set  of  items  for  each  instructional  module. 

6.  Proportion  of  Correct  Answers— The  proportion  of  learners 
who  answered  an  item  correctly  on  the  posttest  is  predicted  to  be 
greater  than  .50. 

7.  Alternatives  for  Test  Items— It  is  predicted  that  on  the 
pretest,  at  least  one  learner  will  select  each  alternative  of  the 
multiple  choice  items. 

8.  Posttest  Performance- “When  the  group  of  learners  are 
divided  into  upper  and  lower  50%,  on  the  basis  of  total  test  score, 
at  least  80%  of  the  learners  in  the  upper  50%  are  predicted  to  answer 
the  item  correctly. 

9.  Fail/Fail  Category  of  Performance— The  mean  proportion 

of  the  learners  in  the  fail/fail  category  of  performance  was  calculated 
for  Group  1  and  Group  2  and  each  was  found  to  approximate  .25.  The 
proportion  of  learners  in  the  fail /fail  category  is  predicted  to  be 
less  than  .25. 

Instructional  modules  and/o**  test  items  are  categorized  as  positive  {*) 
if  there  is  agreement  between  the  observed  and  predicted  div'ectlon  of 
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change.  The  Instructional  modules  and/or  test  Items  are  categorized 
as  negative  (-)  If  there  Is  disagreement  between  the  observed  and 
predicted  direction  of  change.  The  negative  Indicators  are  totaled 
for  each  Instructional  module  and  the  total  is  referred  to  as  the 
Revision  Indicator.  Using  the  performance  data  for  Group  1  and 
Group  2,  Revision  Indicators  were  calculated  for  each  Instructional 
module  (see  Table  15).  There  Is  substantial  agreement  between  the 
rankings  of  the  Instructional  modules  using  the  Revision  Indicators 
derived  from  the  learner  performance  data  for  Group  1  and  Group  2 
(r  *  .83).  The  same  three  Instructional  modules  and  related  test 

'S' 

Items— Measures  of  Central  Tendency,  Pretest/ Posttest,  and  Test  Items— 
were  Identified  as  being  In  need  of  review  and  possible  revision. 

The  Pretest/Posttest  instructional  module  was  the  only  one  of  the 
three  Instructional  modules  identified  which  had  actually  been  revised 
during  Phase  3  of  the  PIERIM  model. 

Another  pt'ornlsing  method  of  analyzing  data  from  a  pretest* 
posttest  design  is  the  McNemar  Test  for  the  significance  of  changes 
(Siegel,  1956).  The  learners'  pretest  and  posttest  performances  can 
be  categorized  for  each  test  item  (see  Table  16)  into  only  one  of  the 
following  categories: 

pretest  -  learned  and  posttest-1 earned  (i.e.,  1/1) 
pretest-learned  and  posttest-unl earned  (i.e.,  1/0) 
pretest-* unlearned  and  posttest-leamed  (i.e.,  0/1),  and 
pretest-unlearned  and  posttest-unleamed  (i.e.,  0/0). 

Learning  was  operationally  defined  as  a  change  from  an  unlearned 
state  on  the  pretest  to  a  learned  state  on  the  posttest  (i.e.,  0/1). 
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TABLE  15. --Revision  Indicators  for  Instructional  Modules 


Instructional  Module 

GROUP  1 

GROUP  2 

Pretest/Posttest 

12 

10 

Behavioral  Objectives 

3 

5 

Test  Items 

11 

10 

Percentile  Ranks 

10 

6 

Measures  of  Central  Tendency 

13 

12 

Normal  Distribution 

5 

8 

Normal  Curve 

2 

1 

Correlation  Coefficient 

3 

2 

Correlatlon/Scatter  Diagram 

4 

5 

Validity 

5 

7 

Rellablllty/Factors  Affecting 

7 

7 

Rel 1 abl 1 1 ty/ In terpretatl on 

6 

6 

Standard  Error  of  Measurement 

2 

3 

Types  of  Tests 

5 

3 

Test  Norms/ Intelligence  Quotient 

7 

7 

Standardized  Test  Information 

5 

7 

Numbers  represent  the  total  number  of  negative  (-) 
Indicators  for  an  Instructional  module 


Group  1  represents  the  19  learners  who  participated  In  Phase  2 
Group  2  represents  the  28  learners  who  participated  In  Phase  4 
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The  McNemar  Test  was  used  to  test  the  significance  of  the  changes 
observed  in  the  learner  performance  data. 

Null  Hypothesis.  for  those  learners  who  change  learning 
states,  the  probability  that  any  learner  will  change  from  an  unlearned 
state  to  a  learned  state  (PQy-j)  Is  equal  to  the  probability  that  any 
learner  will  change  from  a  learned  to  an  unlearned  state  (P-i/q)* 

Alternate  Hypothesis.  H^:  for  those  learners  who  change  learning 
states,  the  probability  that  any  learner  will  change  from  an  unlearned 
state  to  a  learned  state  (Pq^^)  is  greater  than  the  probability 
that  any  learner  will  change  from  a  learned  to  an  unlearned  state  (P^q)* 

Statistical  Test.  The  McNemar  Test  for  the  significance  of 
changes  was  chosen  because  the  study  used  two  related  samples,  was  of 
a  pre'posttest  design,  and  used  nominal  measurement. 

Significance  Level.  Let  alpha  «  .001 

2 

Rejection  Region.  X  greater  than  or  equal  to  5.42. 

The  McNemar  Test  was  computed  for  each  of  the  instructional  modules 
for  which  the  number  of  learners  changing  learning  states  was  greater 
than  ten  (see  Table  17).  The  probability  associated  with  each  value 
calculated  is  reported  for  those  which  did  not  equal  or  exceed  the 
critical  value.  While  the  McNemar  Test  Is  sensitive  to  changes  In 
learning  states,  there  needs  to  be  another  statistic  which  Is  sensi* 
live  to  the  proportion  of  learners  who  remain  In  an  unlearned  state 
on  a  posttest. 
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TABLE  17.--McNeniar  test  for  significance  of  changes 


Instructional  Module  Group  1  Group  2 


Pretest/Posttest 
Behavioral  Objectives 
Test  Items 
Percentile  Ranks 
Measures  of  Central  Tendency 
Normal  Distribution 
Normal  Curve 
Correlation  Coefficient 
Correlation/Scatter  Diagram 
Validity 

Rellablllty/f^actors  Affecting 
Rel labll Ity/ Interpretation 
Standard  Error  of  Measurement 
Types  of  Tests 

Test  Nones/ Intelligence  Quotient 
Standardised  Test  Information 


.02<  p  S.05 

REJECT** 

REJECT** 
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*  Number  of  learners  changing  learning  state 
less  than  10,  HcNemar  Test  Is  not  appropriate. 

**  The  Null  Hypothesis  Is  rejected. 

Group  1  represents  the  19  learners  who  participated  In  Phase  2 
Group  2  represents  the  28  learners  trtio  participated  In  Phase  4 


CHAPTER  IX 


DISCUSSION  OF  RESULTS 

The  problem  and  the  results  of  the  study  are  discussed  in 
relationship  to  the  three  separate  but  related  parts  of  the  study, 
as  follows: 

1.  A  model  for  the  Production,  Implementation,  Evaluation,  and 
Revision  of  Instructional  Modules  (PIERIM)  was  developed  and  a 
detailed  non-emplrlcal  comparison  between  PIERIM  and  two  existing  system 
models  for  design  of  Instruction  was  presented. 

2.  The  Implementation  of  the  PIERIM  model  for  a  portion  of  an 
undergraduate  elementary  education  course:  (a)  established  the  feasibility 
of  Implementing  the  model,  (b)  evaluated  the  learners'  performance 

after  they  Interacted  with  the  Instructional  modules  and  resources, 

(c)  refined  the  job  descriptions  for  the  participants  of  the  Imple* 
mentation  by  reformulating  the  PIERIM  (2nd  edition)  model,  (d)  resulted 
In  recommendations  concerning  the  contents  of  training  materials  that 
could  be  used  to  de^lop  the  skills  required  to  Implement  the  PIERIM 
model,  and  (e)  provided  data  to  estimate  the  cost  of  implementing  the 
model  In  terms  of  design  and  production  of  the  Instructional  modules  and 
tests. 

3.  An  Instructional  Support  System  (ISS)  was  developed  which 
utilized  existing  technology  to  support  the  implementation  of  the  PIERIM 
model.  The  development  was  analyzed  to  determine:  (a)  elements 

of  the  system,  (b)  adequacy  of  the  ISS  as  measured  by  the  output  of 
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the  system,  and  (c)  cost  estimates  in  terms  of  time  and  materials 
required  to  implement  the  ISS. 

Comparison  of  the  System  Models  for 
Design  of  Instruct!^ 

The  comparison  of  Briggs'  (1970)  and  Dick's  (1969)  models  for 
design  of  instruction  with  the  PIERIM  model  identified  three  potential 
weaknesses  in  the  set  of  instructional  modules  produced  as  a  result  of 
implementing  the  PIERIM  model. 

1.  The  relevance  of  the  instructional  modules  could  not  exceed 
the  relevance  of  the  subject  matter  content  when  compared  to  a  specific 
teaching  competency. 

Expert  opinion  was  used  to  establish  the  relevance  of  the  set 
of  Instructional  modules  for  the  evaluation  unit  of  an  undergraduate 
elementary  education  course.  When  faculty  members  from  the  departments 
of  Elementary  Education  and  Educational  Research  ranked  the  set  of 
instructional  rr  dules  designtjd  for  use  in  the  conventional  classroom 
environment  with  respect  to  the  relative  importance  of  each  instructional 
module,  there  was  moderate  agreement  (r^  »  .71)  established  between  the 
ranking  of  the  instructional  modules  by  members  of  the  two  departments. 
The  set  of  25  instructional  modules  was  found  to  represent  a  major 
portion  (i.e.,  50%  to  100%)  of  the  content  that  seven  of  the  eight 
faculty  members  would  include  if  they  were  to  teach  the  evaluation  unit. 

The  relevance  of  the  instructional  modules  has  not  been 
established  for  any  specific  teaching  competency  and  several  of  the 
learners  questioned  the  relevance  of  the  evaluation  unit  for  elementary 
education  majors.  The  learners  expressed  a  desire  for  an  opportunity 
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to  observe  and/or  apply  the  evaluation  skills  in  an  actual  classroom 
situation.  The  relevance  of  the  instructional  modules,  as  perceived 
by  a  group  of  learners,  depends  upon  the  instructor's  ability  to  relate 
the  content  to  specific  teaching  compstencies  and  then  to  communicate 
this  relationship  as  the  purpose  section  of  the  instructional  modules. 

2.  The  instructional  modules,  in  toto,  might  be  insufficient  to 
achieve  a  specified  learner  behavior. 

The  instructor  reported  that  the  learners  who  had  completed  the 
16  instructional  modules  in  the  self-instruction  environment  were  better 
prepared  for  the  remaining  instructional  modules  which  he  taught 
than  were  the  learners  who  had  been  taught  in  the  conventional  classroom 
environment.  The  learners  who  studied  in  the  self-instruction 
environment  needed  less  additional  instruction  on  the  content  covered  by 
the  16  Instructional  modules,  asked  questions  which  demonstrated  their 
understanding  of  the  content,  and  produced  written  assignments  of  a 
higher  quality  for  the  9  instructional  modules  taught  and  evaluated 
by  the  instructor. 

3.  The  instructional  resources  that  were  matched  with  an  instruc¬ 
tional  module  could  be  inadequate  to  achieve  the  desired  learner  behavior. 

If  adequacy  of  an  instructional  module  to  achieve  a  desired 
learner  behavior  is  defined  as  the  proportion  of  the  learners  achieving 
the  stated  standard  of  performance,  then  each  instructional  module  must 
be  evaluated  separately.  The  proportion  of  learners  achieving  the 
stated  standard  of  performance  exceeds  70%  for  10  of  the  16  instructional 
modules  on  the  posttest  (see  Table  11).  There  is  not  a  satisfactory 
method  for  determining  whether  the  resources  identified  for  the 
instructional  modules  were  inadequate  or  whether  the  learners  did  not 


147 


fully  utilize  the  resources  that  were  made  available.  The  original 
set  of  instructional  modules  was  derived  primarily  from  a  content 
analysis  of  a  single  text  (i.e.,  Lindvall,  1967)  and  the  largest  number 
of  learners  consistently  identified  this  same  text  as  the  printed 
resource  they  had  used  to  study  for  the  posttest  (see  Table  9). 

The  problem  of  determining  the  adequacy  of  the  resources  was 
of  sufficient  importance  to  dictate  a  change  in  the  PIERIH  model. 

Between  Phase  3— Revision  of  Instructional  Modules  and  Tests  and  Phase 
4— Implementation  and  Evaluation  of  Instructional  Modules  in  a  Self- 
Instruction  Environment,  procedures  are  recommended  for  individual  and/ 
or  small  group  implementation  of  the  instructional  modules  under 
controlled  conditions.  The  controlled  conditions  will  assure  that  all 
of  the  resources  are  available  to  the  learners  and  that  adequate  time 
is  provided  for  the  learners  to  interact  with  the  resources. 

For  10  of  the  16  Instructional  modules,  the  proportion  of  learners 
achieving  the  stated  standard  of  performance  on  the  posttest  exceeded 
.70.  The  effectiveness  of  the  instructional  modules  did  not  approach 
the  performance  levels  reported  by  studies  which  have  utilized  system 
models  for  design  of  instruction  (Hagerty,  1970;  Mager,  1967;  &  Short, 
et  al.,  1968).  Hopefully  the  effectiveness  of  the  instructional  modules 
produced  through  the  implementation  of  the  PIERIM  model  can  be  Improved. 

It  would  be  unrealistic  to  expect  a  system  model  for  design  of  instruction 
which  is  based  on  selection  of  resources  to  produce  instructional 
materials  which  were  as  effective  as  those  produced  by  system  models 
which  design  instructional  materials  specifically  for  a  set  of 
behavioral  objectives. 
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The  basis  used  to  compare  the  three  system  models  for  design 
of  instruction  is  considered  appropriate  for  reviewing  any  model  for 
design  of  instruction.  The  points  of  comparison  for  the  three  system 
models  are  as  follows: 

1.  Assumptions  upon  which  the  model  is  based. 

2.  Personnel  required  to  implement  the  model. 

3.  Intended  level  of  application. 

4.  Level  of  analysis. 

5.  Activities  required  to  be  completed  in  the  implementation  of 
the  model . 

The  potential  user  of  a  model  for  design  of  instruction  would  want  to 
make  a  more  detailed  analysis  of  any  system  model  for  design  of  instruc¬ 
tion  before  an  attempt  was  made  to  apply  the  model  to  an  instructional 
problem.  At  the  present  time  the  most  complete  description  of  the  steps 
involved  in  the  implementation  of  a  system  model  for  design  of  instruction 
is  Briggs'  (1970)  most  recent  description  of  his  model. 

Implementation  of  the  PIERIM  Model 
The  implementation  of  the  PIERIM  model  is  discussed  as  it  related 
to:  (1)  selection  of  the  instructor,  (2)  planning  requirements, 

(3)  interaction  between  the  instructor  and  educational  technologist, 

(4)  evaluation  of  learner  performance,  (5)  role  of  the  instructor,  (6) 
role  of  the  learner,  and  (7)  role  of  the  educational  technologist. 

Selection  of  the  Instructor 

The  educational  technologist  had  to  actively  solicit  the 
cooperation  of  a  participating  Instructor.  The  problem  of  securing 


the  participation  of  an  instructor  related  in  part  to  the  use  of  an 
untested  system  model  for  design  of  instruction  and  the  use  of  instruc¬ 
tional  procedures  which  did  not  utilize  the  instructor  as  the  primary 
resource.  Instructors  will  probably  continue  to  avoid  empirical 
validation  of  their  instructional  materials  and/or  procedures  until 
there  is  an  identifiable  payoff  for  the  investment  of  their  time  and 
effort. 

The  instructor  who  parti cipatexl  in  the  study  showed  positive  approach 
tendencies  (Mager,  1968b)  toward  the  PIERIM  model  in  that  he  voluntarily 
continued  work  on  the  instructional  modules  beyond  his  original 
commitment  to  the  study.  One  of  the  activities  that  he  wanted  to  complete 
was  the  development  of  a  revised  outline  for  the  Evaluation  Unit.  Prior  to 
designing  the  revised  unit  outline,  the  instructor  and  educational  tech¬ 
nologist  attempted  to  relate  the  Instructional  modules  as  resources 
for  other  instructional  modules  which  were  evaluated  by  the  Instructor. 

The  instructor  stated  that  he  felt  confident  that  he  could  implement 
the  entire  PIERIM  model  by  himself  with  the  exception  of  the  ISS  computer 
program. 


Planning  Requirements 

The  time  schedule  for  InHilementation  of  the  PIERIM  model  (see 
Figure  14)  Indicates  the  starting  and  ending  dates  for  the  selection 
of  the  Instructor  and  each  of  the  four  phases  of  the  model.  The 
Implementation  of  the  PIERIM  model  Indicates  the  need  for  adequate 
planning  on  the  part  of  the  educational  technologist,  and  four  critical 
events  which  were  Identified  are: 
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1.  Selection  of  the  instructor. 

2.  Phase  2— administration  of  the  pretest. 

3.  Phase  4—administration  of  the  pretest. 

4.  Phase  4--administration  of  the  posttest- 

The  dates  of  the  pretest  for  Phases  2  and  4  are  critical  because  the 
instructional  modules  and  tests  must  be  designed  and/or  revised  and 
produced  prior  to  each  of  the  administrations  of  the  pretest.  The 
administration  of  the  posttest  in  the  self-instruction  environment  is 
a  critical  event  because  of  the  coordination  required  for  the  use  of 
the  other  university  facilities  (i.e.,  library,  computer  center  and  test 
service). 

The  Implementation  of  the  PIERIM  model  limited  the  resources  utilized 
to  the  audio  tapes  and  reserve  books.  This  decision  was  based  on  the 
availability  of  the  tape  library  and  reserve  room  on  a  regular  basis 
to  the  Instructional  staff  at  The  Florida  State  University.  The  test 
scoring  service  and  the  computer  facilities  utilized  by  the  ISS  are 
similarly  available  to  the  instructional  staff. 

Interaction  Befteen  the  Instructor 
and  bducatlonal  YechnologUt 

The  Interaction  between  the  Instructor  and  educational 
technologist  was  guided  primarily  by  the  instructional  module  worksheet 
(see  Figure  15).  The  productivity  of  the  work  sessions  was  found  to 
be  related  to  the  educational  technologist's  preparation  for  the  session. 

A  greater  amount  of  Information  was  exchanged  during  a  work  session 
when  the  educational  technologist  asked  the  instructor  specific  questions 
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CONTENT  CLASSIFICATION: 


! 


PURPOSE: 


BEHAVIORAL  OBJECTIVE: 


SAMPLE  TEST  ITEM: 


RESOURCES: 


Fig.  15— •Instructional  module  worksheet 


rather  than  conducting  an  unstructured  work  session.  A  prototypic 
set  of  questions  which  the  educational  technologist  would  ask  the 
Instructor  would  Include: 

1.  What  competencies  are  you  going  to  develop  In  relation  to 
the  topic  described  by  the  Content  Classification? 

2.  What  Is  the  purpose  of  the  learner  mastering  this  competency? 

3.  How  would  you  determine  whether  a  learner  has  mastered  the 
competency? 

4.  What  resources  would  you  use  to  teach  the  competency  to  a 
learner? 

5.  The  last  time  we  met,  you  stated  that  the  learner  should 
develop  this  coii4)etency.  Does  the  behavioral  objective  that 
I  have  written  describe  the  competency  as  you  described  It? 

6.  Can  the  learner's  ability  to  perform  the  competency  be 
evaluated  by  the  sample  test  Item? 

7.  Does  the  purpose  adequately  describe  the  reasons  for  a 
learner  mastering  the  behavioral  objective? 

This  series  of  questions  was  quite  satisfactory  for  refining  the  set 

of  Instructional  modules  which  were  evaluated  by  the  multiple  choice 

test  Items.  The  educational  technologist  was  generally  unsuccessful 

In  getting  the  Instructor  to  reveal  the  criteria  which  he  was  using 

to  evaluate  the  learner's  performance  on  the  remaining  nine  Instructional 

modules. 

Both  the  faculty  members,  who  evaluated  the  Instructional  mo<hi1as» 
and  the  learners  generally  agreed  that  the  sa^»1e  test  Items  measured 
the  stated  behavioral  objectives.  One  of  the  weaker  elements  of  the 
Interaction  between  the  Instructor  and  educational  technologist  and 
subsequently  the  Instructional  modules  was  the  description  of  the  hirpose 
portion  of  the  modules.  The  learners,  who  participated  In  the 
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implementation  of  the  instructional  modules  in  the  conventional  classroom 
environment,  were  critical  of  a  perceived  lack  of  relevance  between  the 
instructional  modules  and  the  learners'  interpretation  of  the  competencies 
required  of  an  elementary  school  teacher.  A  greater  effort  to  relate 
the  instructional  modules  to  specific  teaching  competencies  would  help 
establish  the  face  validity  of  the  instructional  modules. 

The  final  form  of  the  instructional  module  contained  the  following 
five  categories  of  information: 

1.  Content  Classification— Key  words  which  can  be  utilized  to  locate 
additional  information  related  to  the  instructional  module. 

2.  Purpose— A  description  of  the  behavior  which  the  instructional 
module  is  designed  to  produce. 

3.  Behavioral  Object! ve*-Oescribes  an  observable  learner  behavior, 
method  of  evaluation,  subject  matter  content,  and  the  standard 
of  performance. 

4.  Sample  Test  Item— Provides  a  san4>1e  of  the  type  of  test  item 
which  will  be  used  to  evaluate  the  learner's  performance. 

5.  Resources— Provide  complete  identification  of  the  resources 
which  have  been  identified  for  the  instructional  module.  The 
specific  pages,  which  are  relevant  to  the  stated  behavioral 
objective,  are  cited  for  each  resource. 

Evaluation  of  Learner  Performance 

The  learners  and  Instructor  were  generally  satisfied  with  the 
siamary  of  pre*  and  posttest  performance  data  which  th^  were  given 
after  each  of  the  tests.  Two  specific  weaknesses  related  to  the 
evaluation  of  learner  performance  were  identified: 

1.  The  same  test  was  used  as  both  the  pre>  and  post test. 

2.  Learner  performance  on  the  pre>  and  posttest  was  used  only 
for  revision  of  the  instructional  materials  and  the  assignment 
of  grades. 
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The  same  form  of  the  test  was  used  for  the  pre-  and  posttest  because  of 
the  recognized  difficulty  of  w»'iting  multiple  test  items  which  measure 
the  learners'  performance  for  a  single  behavioral  objective.  Additional 
test  items  were  included  on  each  posttest  to  determine  if  the  new  items 
did  in  fact  demonstrate  item  characteristics  which  were  similar  to  the 
characteristics  of  the  original  test  items  that  were  used  to  evaluate 
the  learners*  performance.  During  the  first  revision  of  the  instructional 
modules  and  tests,  the  additional  items  were  a  valuable  source  of 
replacement  items.  The  goal  of  the  instructor  and  educational  technologist 
should  be  the  development  of  at  least  two  equivalent  forms  of  the  tests 
for  each  instructional  module. 

The  research  design  should  be  changed  from  a  One  Group  Pretest- 
Posttest  Design  to  include  a  control  group  which  does  not  receive  the 
Instructional  treatment.  This  would  provide  a  means  for  determining 
whether  the  pretest-posttest  performance  gains  can  be  attributed  to  the 
instructional  modules  and/or  procedures,  or  if  the  gains  should  be 
attributed  to  the  effects  of  testing. 

The  learners  during  Phase  4— Implementation  and  Evaluation  of 
Instructional  ik>dules  in  a  Self-Instruction  Environment  reported  that 
they  used  their  performance  on  the  pretest  to  guide  their  study  efforts 
for  the  posttest.  The  learners  who  achieved  the  standard  of  performance 
for  an  instructional  module  on  the  pretest  were  not  exempt  from  the 
instruction  related  to  the  instructional  module  but  rather  were  tested 
again  on  the  total  set  of  instructional  modules.  The  adniftistration  of 
a  total  unit  test  for  both  the  pre-  and  posttest  iMS  chosen  bKause  it 
greatly  simplified  the  testing  procedures  and  the  interpretation  of 
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the  perfonnance  data.  The  instructor  and  educational  technologist  were 
not  that  confident  in  either  the  test  items  or  the  standard  of  performance 
that  they  thought  that  learners  should  be  exempt  from  instruction. 

The  ISS  should  be  modified  to  support  an  individualized  testing 
program  which  would  exempt  learners  from  instructional  modules  on  the 
basis  of  the  learner's  pretest  performance  and  prepare  posttests  for 
only  those  instructional  modules  on  which  the  learner  failed  to  achieve 
the  stated  standard  of  performance.  There  are  other  configurations  of 
an  ISS  that  could  easily  accotmodate  this  change  such  as  a  computer- 
assisted  instruction  system.  The  testing  procedures  can  be  totally 
individualized  utilizing  the  existing  capabilities  of  the  IBM  1500 
system  and  the  COURSEURITER  II  programming  language. 

PIERIM  Model  (2nd  Edition) 

The  PIERIM  Model  (2nd  Edition)  for  design  of  instruction  involves 
five  phases  of  systematic  interaction  between  an  instructor  and  an 
educational  technologist  for  the  purpose  of  designing  instructional 
modules  for  use  in  a  self-instruction  environment.  The  following 
explication  of  the  PIERIM  Model  (2nd  Edition)  is  divided  into  the  five 
phases: 

Phase  1— Design  cf  Instructional  Modules 

Phase  2— Implementation  and  Evaluation  of  Instructional  Modules 
in  a  Conventional  Environment 

Phase  3— Revision  of  Instructional  Modules  and  Tests 

Phase  4— Individual  and/or  Small  Group  Field  Test  of  the 
Instructional  Modules 

Phase  5— Implementation  and  Evaluation  of  Instructional  Modules 
in  a  Self-Instruction  Environment, 
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The  PIERIM  Model  {2nd  Edition)  refines  the  set  of  activities  and 
operationally  defines  the  job  descriptions  for  the  Instructor  (I)  and  the 
Educational  Technologist  (T).  The  person  responsible  for  accomplishing 
each  of  the  stated  activities  is  identified  as  either  the  Instructor  (I) 
or  the  Educational  Technologist  (T): 

Phase  1— Design  of  Instructional 
Modules 

(I)  Select  content  area. 

(I)  Write  a  detailed  outline  of  the  content  area. 

(I)  Verbally  state  learner  competencies  to  be  developed. 

(I)  Verbally  state  the  purpose  for  a  learner  achieving  each  competency. 
(T)  Write  behavioral  objectives  for  each  competency. 

(T)  Categorize  each  behavioral  objective  within  the  content  outline. 

(T)  Write/Select  sample  test  items  for  each  behavioral  objective. 

(I)  Evaluate  behavioral  objectives  and  test  items. 

(I)  List  resources  for  each  behavioral  objective. 

(T)  Write  Instructional  Modules  for  each  behavioral  objective. 

(T)  Write  evaluation  instruments. 

Phase  2— Implementation  and 
Evaluation  of  Instructional 
Modules  in  a  Conventional  ~ 

Classroom  fenvironmenT 

(I)  Administer  pretest. 

(T)  Prepare  summary  of  learner  performance  on  pretest. 

(I)  Implement  instructional  modules  in  a  conventional  classroom 
environment. 

(T)  Prepare  detailed  notes  and  outline  of  class  lectures. 
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(I)  Administer  posttest. 

(T)  Prepare  summary  of  learner  performance  on  posttest. 


Phase  3-~Revision  of  Instructional 

Nodules  and  tests 

(T)  Identify  changes  which  affect  all  of  the  instructional  modules. 

(I)  Specify  changes  to  be  made  to  instructional  modules  and/or 
test  items  based  on  a  review  of  learner  performance  data. 

(T)  Identify  additional  resources  for  the  instructional  modules. 

(T)  Match  resources  to  each  individual  Instructional  module. 

(I)  Review  class  notes  and  additional  resources  for  each  instructional 
module. 

(I)  Prepare  a  summary  for  each  instructional  module. 

(I)  Prepare  an  audio  tape  of  the  summary  for  each  instructional 
modul e , 

(T)  Revise  instructional  modules. 

(T)  Revise  evaluation  exercises. 


Phase  4— Individual  and/or  Small 
Field  Test  of  Instructional 
_ « 

(T)  Administer  pretest  for  instructional  modules. 

(T)  Provide  learner(s)  with  each  applicable  resource. 

(T)  Admini?. ler  evaluation  of  resources. 

(T)  Administer  posttest  for  instructional  modules. 

(T)  Prepare  summary  of  learner  performance. 

(I)  Interview  learner  and  discuss  his  performance  on  the 
instructional  modules. 
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Phase  5-- Implementation  and  Evaluation 
of  instructional  Modules  in  a  Self- 
Instruction  Environment 

(I)  Administer  pretest. 

(T)  Prepare  summary  of  learner  performance  on  pretest. 

(I)  Implement  Instructional  Modules  in  self-instruction 
environment. 

(I)  Administer  pcsttest. 

(T)  Prepare  summary  of  learner  performance  on  posttest. 

(I  &  T)  Repeat  Phase  3  through  Phase  5,  until  each  Instructional 
Module  achieves  the  standard  set  for  the  instructional 
system. 

A  flow  diagram  (see  Figure  16)  represents  the  five  phases  of  the  PIERIM 
Model.  Just  as  the  single  implementation  of  the  PIERIM  Model  resulted 
in  changes  to  the  model,  future  implementations  of  the  model  will  be 
expected  to  further  clarify  the  activities  and  possibly  identify  addi¬ 
tional  activities  which  must  be  included  in  the  PIERIM  Model. 

The  PIERIM  Model  (2nd  Edition)  does  not  differ  sufficiently 
from  the  original  version  of  the  PIERIM  Model  to  affect  the  comparison 
of  the  model  with  Briggs'  (1970)  and  Dick's  (1969)  models.  The  major 
change  included  in  the  PIERIM  Model  (2nd  Edition)  is  Phase  4  -  Individual 
and/or  Small  Group  Field  Test  of  Instructional  Modules.  The  PIERIM 
Model  (2nd  E-lition)  provides  the  means  of  minimizing  all  but  the  first 
of  the  three  potential  weaknesses  in  a  set  of  instructional  modules 
produced  as  a  result  of  implementing  the  model: 


PHASE  1 

DESIGN  OF  INSTRUCTIONAL  MODULES 


-A  model  for  production,  implementation,  evaluation, 
and  revision  of  instructional  modules  (PIERIM)  - 
2nd  edition 
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1.  The  relevance  of  the  instructional  modules  cannot  exceed  the 
relevance  of  the  subject  matter  content  when  compared  to  a  specific 
teacher  behavior. 

2.  The  instructional  modules,  in  toto,  might  be  insufficient 
to  achieve  a  specified  learner  behavior. 

3.  The  instructional  resources  that  are  matched  with  an  instruc¬ 
tional  module  could  be  inadequate  to  achieve  the  desired  learner  behavior. 

Role  of  the  Instructor 

The  instructor  is  the  content  expert  and  final  authority  in  the 
PIERIM  model.  He  is  responsible  for  defining  the  learner  competencies 
to  be  developed,  the  manner  in  which  the  competencies  are  to  be  evaluated, 
and  the  resources  to  be  used  to  develop  the  competencies.  All  of  the 
work  of  the  educational  technologist  is  reviewed  and  either  approved  or 
modified  by  the  instructor.  The  instructor  interacts  with  the  learners 
for  the  purpose  of: 

1.  administering  the  pre-  and  posttest. 

2.  orienting  the  learners  to  the  instructional  modules  and  procedures. 

3.  serving  as  either  a  primary  or  secondary  resource  for  each  instruc¬ 
tional  module,  and 

4.  evaluating  the  learners'  performance  for  each  instructional 
module  which  specifies  evaluation  by  the  instructor. 

The  method  used  to  document  the  interaction  between  the  instructor  and 
the  learners  (i.e.,  tape  recording  of  class  lectures)  proved  to  be 
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unsatisfactory  due  to  the  large  number  of  equipment  and/or  human 
errors  associated  with  the  activity.  The  educational  technologist  was 
made  responsible  for  the  documentation  of  the  interaction  in  the  class¬ 
room  in  the  PIERIM  Model  (2nd  Edition).  The  use  of  a  classroom  observer 
will  be  the  method  next  used  to  document  the  interaction  between  the 
instructor  and  the  learners. 

It  is  perhaps  unwise  to  generalize  from  a  single  implementation 
of  the  PIERIM  model  because  the  specific  activities  and/or  the  sequence 
of  activities  could  conceivably  change  if  the  model  were  implemented 
with  an  instructor  who  was  more  experienced  in  teaching  the  course  being 
developed.  Based  on  a  single  implementation  of  the  PIERIM  Model,  future 
implementation  of  the  PIERIM  Model  (2nd  Edition)  would  be  facilitated 
if  the  Instructor  were  given  training  in  the  following  areas: 

1.  stating  desired  learner  competencies  in  terms  of  observable 
behaviors* 

2.  relating  content  mastery  to  future  teaching  competencies,  and 

3.  interacting  with  existing  educational  technology. 

Role  of  the  Learner 

The  learners  were  given  three  distinct  roles  within  the  implemen¬ 
tation  of  the  instructional  modules.  The  learners: 

1.  Interacted  with  the  instructional  modules, 

2.  recommended  changes  to  the  Instructional  modules  and/or 
procedures,  and 

3.  expressed  their  attitude  toward  the  instructional  modules 
and/or  procedures. 
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The  learners  did  not  appear  to  understand  that  they  could  make  a  meaning¬ 
ful  contribution  to  the  revision  of  the  instructional  modules.  One  of 
the  major  tasks  related  to  the  implementation  of  the  PIERIM  Model  (2nd 
Edition)  will  be  the  development  of  methods  and  procedures  for  the  orien¬ 
tation  of  learners  to  the  importance  of  empirical  validation  of  instruc¬ 
tion  and  their  role  in  the  validation  procedures. 

The  use  of  the  Evaluation  of  Resources  (see  Figure  13)  resulted 
in  an  improved  level  of  learner  participation  in  the  PIF.RIM  Model.  The 
simplicity  of  the  form  and  the  ease  with  which  the  learners  could 
respond  are  considered  the  major  reasons  for  the  improved  participation 
on  the  part  of  the  learners.  The  Student  Questionnaire  provided  a  means 
by  which  the  learners  could  respond  to  specific  aspects  of  the  instruc¬ 
tional  modules  and  procedures.  The  questionnaire  needs  to  be  redesigned 
using  a  more  structured  form  of  questioning  and/or  specified  categories 
of  responses  from  which  the  learner  could  select  rather  than  construct 
his  response.  The  aim  of  the  questionnaire  is  to  provide  a  conmuni cation 
channel  between  the  learners,  instructor  and  educational  technologist 
which  is  readily  and  unambiguously  interpreted. 

The  training  recommended  for  the  learners  includes  the  opportunity 
to  participate  in  instructional  development  activities  on  a  regular  and 
required  basis.  Their  preparation  for  participation  in  the  development 
activities  should  include: 

1.  system  models  for  design  of  instruction, 

2.  demonstration  of  instructional  methods  and/or  procedures,  and 

3.  introduction  to  educational  research  methodology. 
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Role  of  the  Educational  Technologist 

The  educational  technologist  is  required  to  coordinate  the 
total  set  of  activities  related  to  the  implementation  of  the  PIERIi-l  Model. 
The  educational  technologist  is  assimed  to  have  mastered  the  skills 
required  to  implement  a  system  model  for  design  of  instruction  such  as 
Briggs'  (1970)  and  Dick's  (1969)  models.  In  addition  to  these  skills, 
the  educational  technologist  should  receive  training  in: 

1.  operation  of  existing  educational  technology, 

2 .  computer  programmi ng , 

3.  project  management  techniques, 

4.  identification  of  sources  of  information  concerning  available 
multimedia  resources,  and 

5.  non  parametric  statistics. 

Cost  of  Implementing  the  PIERIM  Model 

The  cost  of  implementing  the  PIERIM  model  reflects  only  the 
time  other  than  actual  class  time  that  the  instructor  and/or  educational 
technologist  were  involved  in  activities  directly  related  to  the  imple¬ 
mentation  of  the  PIERIM  model.  The  personnel  costs  are  lower  than  one 
would  expect  because  both  the  instructor  and  educational  technologist 
are  figured  at  $4.00  per  hour  or  $320  per  month  for  a  half  time  graduate 
assistant.  If  the  personnel  costs  for  design  of  the  instructional  modules 
and  tests  were  doubled,  $8.00  per  hour,  then  this  would  represent  an 
annual  salary  of  $ii.,360  for  both  the  instructor  and  the  educational  tech¬ 
nologist.  A  summary  of  the  actual  cost  of  implementing  PIERIM  (see 
Table  18)  indicate  that  the  total  cost  of  designing  and  producing  the  30 


sets  of  the  25  Instructional  modules  and  tests  is  $768.52  or  an  average 
of  $25.80  per  instructional  module  and  related  test  items.  The  design 
and  production  cost  based  on  an  annual  salary  of  $15,360  would  increase 
to  $49.28  per  instructional  module  and  test  items.  The  projected  cost 
of  implementing  the  PIERIM  Model  with  faculty  ment)ers  (see  Table  19) 
estimates  the  cost  of  designing  1.  25.  and  100  instructional  modules 
and  test  items  and  the  production  of  1,  30.  100.  and  300  sets  of  the 
instructional  modules  and  test  items. 

TABLE  19.— Cost  projections  for  the  design  and  production  of  instructional 
modules  and  test  items 


Number  of  Instructional 

Modules  Desioned 

Number  of  Sets  Produced 

I 

30 

105 

300 

1 

$  49.28 

$  54.21 

$  66.11 

$  100.11 

25 

1232.00 

1355.25 

1652.75 

2502.75 

100 

4928.00 

5421.00 

6611.00 

10.011.00 

NOTE:  Cost  estimates  are  based  on  the  1iq>1ementation  of  the  PIERIM 
Model  with  faculty  meirtbers.  The  desim  cost  and  production 
cost  figures  are  $49.11  and  $.17  per  instructional  module  and 
associated  test  items. 

Instructional  Support  System 

The  Instructional  Support  System  (ISS)  is  operationally  defined 
as  the  set  of  capabilities  required  to  support  the  implementation  of 
the  PIERIM  Model.  The  set  c:-^uter  programs  (see  Appendix  G)  and 
the  other  resources  that  comprised  the  ISS  represent  just  one  of  many 
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possible  configurations  of  an  ISS  capable  of  supporting  the  implementa¬ 
tion  of  the  PIERIM  Model.  The  ISS  serves  three  functions: 

1.  production  of  instructional  modules  and  tests* 

2.  evaluation  of  learner  performance*  and 

3.  analysis  and  summary  reports  of  learner  performance  data. 

The  specific  capabilities  required  of  the  ISS  for  the  production  of  the 
instructional  modules  and  tests  include: 

1.  production  of  any  specified  number  of  sets  of  the  instructional 
modules  and  tests, 

2.  sequencing  of  the  instructional  modules  and  test  items  In  any 
specified  order*  and 

3.  revision  of  the  instructional  modules  and  test  items  possible 
at  the  lowest  practical  level. 

The  ISS  programs  IMPROO  and  TEST  possess  each  of  the  capabilities 
specified  with  the  line  of  print  being  used  as  the  basic  unit  for  revision 
of  the  instructional  modules  and  test  items. 

The  cost  of  producing  the  ISS  computer  programs  camot  be  pre¬ 
cisely  determined.  The  educational  technologist  progranaed.  key  punched 
and  tested  each  of  the  programs.  The  best  estimate  of  the  total  cost 
of  producing  and  testing  the  programs  Is  $434.00.  The  cost  of  producing 
sets  of  the  instructional  modules  and  tests  are  based  on  production 
runs  of  30  sets  of  each  of  the  materials.  The  production  runs  utilized 
binary  source  decks  and  program  listing  were  not  produced.  The  suimuiry 
reports  of  the  learners  performance  are  based  on  the  actual  production 
costs  associated  with  the  implementation  of  the  programs  during 
Phase  4  cf  the  PIERIM  Model. 
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The  Division  of  Instructional  Research  and  Service  at  The 
riorlda  State  University  offers  a  test  scoring  and  analysis  service 
for  use  by  the  Instructional  staff.  The  characteristics  required  for 
the  evaluation  of  the  learner's  performance  Include  methods  for: 

1.  presenting  the  test  Items, 

2.  recording  the  learner's  responses, 

3.  evaluating  the  learner's  responses, 

4.  summarizing  the  learner's  responses,  and 

5.  producing  a  record  of  the  learner's  responses. 

The  test  scoring  service  provided  the  means  of  accomplishing  all  of  the 
requirements  with  the  exception  of  presenting  the  test  Items.  The  summary 
of  the  learner's  responses  was  for  the  total  test  rather  than  for  each 
Instructional  module  separately.  The  response  record  of  each  learner 
was  punched  In  a  format  which  was  Interpretable  by  a  computer  program 
written  In  FORTRAN  IV  programming  language  and  the  ISS  programs  EVAL  and 
STAT  provided  more  detailed  analysis  of  the  learner's  performance. 

The  services  provided  by  the  university  library  Included:  place- 
lent  of  the  books  Identified  as  resources  for  the  Instructional  modules 
In  the  reserve  room  of  the  library,  storage  of  the  audio  tapes,  and 
facilities  for  listening  to  the  tapes.  The  library  provided  the  Instructor 
with  a  monthly  report  of  the  tape  usage  by  tape  number  and  time  of  day 
(I.e.,  day  or  nl^t).  The  set  of  11  tapes  were  checked  out  a  total  of 
263  tlMS  with  28  percent  of  the  usage  occurring  after  6  p.m.  and  56 
percent  of  the  usage  occurring  the  day  preceding  the  posttest.  The 
audio  tapes  were  produced  as  summaries  for  the  Instructional  modules 
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and  the  learners  generally  found  them  to  be  a  useful  resource  In  pre¬ 
paring  for  the  posttest. 

The  ISS  supported  the  implementation  of  the  instructional 
modules  and  tests  in  the  self-instruction  environment.  There  were  never 
any  delays  in  the  program  due  to  the  ISS>  and  the  learners  received 
the  summary  of  their  performance  of  the  pretest  and  posttest  the  day 
after  they  completed  the  test.  The  main  disadvantage  associated  with 
the  ISS  is  the  relatively  high  cost  of  using  the  computer  as  the  repro¬ 
duction  facility  for  the  instructional  modules  and  tests.  It  is  possible 
that  an  alternative  strategy  could  be  implemented;  using  the  computer  to 
generate  a  master  copy  and  then  reproducing  the  multiple  copies  by  more 
traditional  methods.  The  major  advantage  of  the  ISS  computer  programs 
is  the  facility  with  which  the  instructional  modules,  tests,  and  reports 
can  be  tailored  to  the  individual  instructor's  specifications. 

Implications  of  the  Study 

This  study  adds  to  some  but  not  all  of  the  areas  of  research 
reviewed  in  Chapter  II  -  Survey  of  Literature.  The  relationship  and 
contribution  of  this  study  to  the  state  of  the  art  are  reviewed  for: 

1.  Current  Interest  in  Curriculum, 

2.  Instructional  Systems, 

3.  Mastery  Models  and  Criterion-Referenced  Measures, 

4.  -Educational  Technology;  and 

5.  Innovations  in  Education. 
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Current  Interest  in  Curriculum 

The  set  of  instructional  modules  produced  through  the  implenienta- 
tion  of  the  PIERIM  Model  for  the  evaluation  unit  cinnot  be  considered 
a  curriculum  for  the  unit  ot  instruction  The  instructional  modi;‘es 
provide  a  statement  of  the  terminal  ob.iect’ves  but  fail  to  either 
describe  the  sequence  of  prerequisite  capabilities  or  identify  the 
initial  capabilities  assi.':r,ed  to  be  possessed  by  the  learner,  it 
conceivable  that  future  revisions  to  the  FIERiM  model  could  resi/i' 
the  production  of  instructional  modules  which  meet  the  requirements 
for  a  curriculum  of  the  content  area  to  which  the  PIERIM  Model  was 
applied. 

Instructional  Systems 

Each  of  the  phases  of  the  PIERIM  model  and  each  of  the  iSS  com¬ 
puter  programs  were  found  lo  correspond  to  at  least  one  of  the  fi-e 
general  subsystems  of  an  instructional  support  system  (see  Figure  i i • 

1.  Phase  1  of  the  PIERIM  model  and  the  ISS  programs  IMPROD  and 

TEST  correspond  to  the  Design/Selection  of  Instructional  Materials 
Subsystem, 

2.  Phase  2  and  Phase  4  of  the  PIERIM  model  correspond  to  the 
Implementation  subsystem, 

3.  ISS  programs  EVAL  and  STAT  correspond  to  the  Evaluation 
Subsystem, 

4.  Phase  3  of  the  PIERIM  model  corresponds  to  the  Revision  Sub¬ 
system,  and 

5.  ISS  programs  EVAL  and  STAT  and  the  interface  established 
between  the  test  scoring  service  and  the  ISS  programs  represents  a 
weak  correspondence  to  the  Information  Management  Subsystem. 
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Although  the  ISS  developed  for  this  study  is  a  relatively  simple  system, 
it  does  relate  to  a  more  general  instructional  support  system  sufficiently 
to  be  referred  to  as  an  ISS.  The  major  advantage  of  the  ISS  designed 
to  support  the  implementation  of  the  PIERIM  model  is  that  only  existing 
technology  was  utili2ed  and  that  technology  is  generally  available  to 
the  staffs  of  teacher  training  programs. 

System  Models  for  Design  of  Instruction 

The  PIERiH  -tccel  was  proved  capable  of  transforming  the  content 
of  an  elementary  eoucation  course  into  a  format  (i.e.,  instructional 
module)  which  is  compatible  with  an  individualized  teacher  training 
program.  The  implementation  of  the  PIERIM  model  was  accomplished  with 
the  assistance  of  «n  instructor  from  the  Department  of  Elementary  Educa¬ 
tion  at  The  Florida  State  University,  Elementary  education  majors 
interacted  with  the  inst>'uctional  modules  and  resources  and  performance 
indicates  that  the  instructional  modules  and/or  procedures  are  moderately 
effective  in  producing  the  desired  behavioral  changes  in  the  learners. 

The  major  modification  included  in  the  PIERIM  model  (2nd  Edition)  is 
the  inclusion  of  the  procedures  for  individual  and/or  small  group  field 
test  of  the  instructional  modules. 

The  PIERIH  model  is  a  contribution  to  the  field  of  system  models 
for  design  of  instruction  because: 

1.  PIERIM  is  a  selection  model  as  opposed  to  a  design  model,  and 

2.  PIERIM  is  an  Interactive  model  which  specifies  activities  for 

the  instructor  (i.e.,  content  expert)  and  educational  technologist 
for  the  implementation  of  the  model. 
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The  assumptions  upon  which  the  PIERIM  model  is  based  are  specified  and 
the  potential  user  of  the  model  can  decide  for  himself  if  the  PIERIM 
model  is  applicable  to  his  instructional  problem.  The  new  assumptions 
are: 

1.  Performance-based  teacher  training  programs  require  a  variety 

of  instructional  systems,  one  of  which  could  be  concerned  primarily 
with  enabling  learners  to  master  relevant  cognitive  skills. 

2.  A  significant  portion  of  the  cognitive  skills  which  comprise 
existing  pre-service  teacher  training  programs  is  applicable  in  a 
performance-based  teacher  training  program. 

3.  The  instructor  is  a  subject  matter  expert. 

4.  The  instructor  is  primarily  a  selector  rather  than  a  designer 
of  instruction,  and 

5.  Existing  system  models  are  best  suited  for  design  of  instruction 
rather  than  selection  of  instruction. 

The  comparison  of  the  PIERIM  model  with  Briggs'  and  Dick's  models 
provides  the  teacher  educators  with  a  frame  of  reference  that  is  appli¬ 
cable  for  reviewing  other  methods  for  designing  instruction: 

1.  assumptions  upon  which  the  model  is  based, 

2.  personnel  required  to  implement  the  model, 

3.  intended  level  of  application, 

4.  level  of  analysis,  and 

5.  activities  required  to  be  completed  in  the  implementation 
of  the  model . 

The  PIERIM  model  could  be  considered  as  a  potential  method  of 
training  faculty  members  in  the  use  of  a  systems  model  for  design  of 
instruction.  After  the  study  was  completed,  the  instructor  stated  that 


he  felt  confident  that  he  could  Implement  the  PIERIM  model  by  himselr 
if  an  educational  technologist  would  assist  with  the  Instructional 
Support  System.  It  is  estimated  that  one  educational  technologist 
could  work  with  three  or  four  faculty  members  simultaneously  in  the 
implementation  of  the  PIERIM  model  if  the  units  of  instruction  were 
not  more  than  three  weeks  in  length. 

Mastery  Models  and  Criterion- 
Referenced  Measures _ 

The  ISS  programs  EVAL  and  STAT  were  specifically  designed  for 
use  with  criterion  referenced  measures  and  a  pretest  -  posttest  evalua¬ 
tion  design.  Preliminary  work  related  to  the  development  of  a  Revision 
Indicator  provides  one  method  of  ranking  instructional  modules  which 
are  evaluated  using  criterion-referenced  measures.  The  McNemar  test 
appears  to  be  useful  for  evaluating  changes  in  learner  performance  from 
pretest  to  posttest.  The  McNemar  test  is  insensitive  to  the  number  of 
learners  who  remain  in  an  unlearned  state  on  the  posttest. 


Instructional  Packages 

The  instructional  modules  produced  through  the  implementation  of 
the  PIERIM  model  do  not  meet  all  of  the  characteristics  of  an  instructional 
package  (Ubben,  1970).  The  instructional  modules  are  best  classified 
as  learning  guides  rather  than  instructional  packages.  The  specific 
characteristics  which  prevent  the  instructional  modules  from  being 
classified  as  instructional  packages  are: 
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1.  Each  lea»'ner  did  not  receive  a  complete  set  of  the  instructional 
modules  and  resciKces  nor  were  there  complete  sets  of  the  materials 
packaged  for  use  by  the  learners  for  each  instructional  module. 

2.  The  media  choice  was  limited  to  O'-'rited  materials  and  audio 
tapes  for  the  instructional  moduie;; 

3.  Evaluation  devii.,es  and  sel  t  tests  .ere  not  mada  available  to 
the  learner  on  the  instructional  mooole  level . 

Edurational  Technology 

The  ISS  programs  and  the  computer  provided  a  means  for  producing 
instructional  modules  and  tests.  Even  though  the  cost  is  greater 
than  producing  the  materials  by  traditional  methods,  the  advantages  of 
speed  and  ease  of  revision  the  materials  make  the  methodology  competitive 
with  traditional  means.  The  possible  solution  of  utilizing  the  computer 
to  produce  a  master  copy  and  then  reproduce  additional  copies  by  a  more 
traditional  means  could  possibly  reduce  the  cost  and  yet  retain  the 
advantages  of  the  computer  generated  instructional  modules  and  tests. 

The  method  of  programming  each  instructional  module  and  test  item  as  a 
separate  FORTRAN  IV  subroutine  greatly  increased  the  flexibility  of  the 
ISS  programs. 

The  popularity  of  the  audio  tapes  cannot  be  interpreted  as 
meaning  that  Lhe  tapes  were  necessarily  effective  in  producing  the  desired 
changes  in  learner  behavior.  The  popularity  of  the  audio  tapes  supports 
the  findings  of  Meene,  et  al.  (1969)  in  that  the  general  reaction  of 
the  learners  to  the  tapes  was  favorable. 
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Innovations  in  Education 

The  problems  associated  v/ith  the  implementation  of  innovative 
changes  in  an  instructional  program  of  higher  education  (Ilayhew,  1967) 
were  encountered  in  the  present  study.  The  most  critical  activity  of 
the  study  was  securing  the  cooperation  of  the  instructor  for  the 
implementation  of  the  PlERIIl  model.  Faculty  members  of  the  teacher 
training  program  appeared  to  be  resistant  to  changes  in  their  instruc¬ 
tional  program  and/or  procedures.  Teacher  training  programs  should 
consider  the  potential  of  utilizing  advanced  graduate  students  to 
prepare  the  instructional  modules  and  tests  required  to  support  an 
individualized  preservice  teacher  training  program.  The  use  of  the  grad¬ 
uate  student  reduces  the  personnel  costs  to  a  minimum. 


CHAPTER  X 


IMPLICATIONS  FOR  FUTURE  RESEARCH 

The  Implications  of  the  study  for  future  research  are  reviewed 
as  they  relate  to  the  areas  of:  teacher  training  systems,  Implementa¬ 
tion  of  the  PIERIM  model ,  and  the  Instructional  Support  System.  There 
are  many  more  major  research  questions  related  to  the  areas  Indicated 
but  the  discussion  will  be  limited  to  those  areas  which  would  be 
beneficial  specifically  to  future  Implementations  of  the  PIERIM  Model 
(2nd  Edition). 


Teacher  Training  Systems 

The  Implementation  of  the  PIERIM  model  focused  on  a  need  for 
research  related  to  both  teacher  training  systems  and  the  PIERIM  model . 
Teacher  training  systems  seemingly  have  not  set  up  procedures  and  reward 
systems  which  would  encourage  Instructors  and/or  learners  to  participate 
In  the  empirical  validation  of  Instructional  materials  and/or  procedures. 

The  need  exists  for  research  related  to  the  development  of  procedures  and 
reward  systems  which  place  an  emphasis  on  the  design  and  validation  of 
Instructional  equal  to  that  currently  placed  on  other  research  activities 
and  publications. 

One  set  of  procedures  which  might  be  considered  Is: 

1.  Establish  within  the  teacher  training  program  the  facilities  and 
staff  of  educational  technologists  to  assist  faculty  members  with  the 
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development  of  instructional  materials  and/or  procedures  for  an  Indivi¬ 
dualized  teacher  training  program. 

2.  Require  that  all  education  majors  participate  on  a  regular  basis 

in  the  development  of  instructional  materials  and/or  procedures  for  the 
teacher  training  program. 

3.  Release  instructors  from  regular  teaching  assignments  for  the 
expressed  purpose  of  designing  instructional  materials  in  their  areas 
of  specialization,  and 

4.  Include  the  development  and  documentation  of  the  effectiveness 

of  instructional  materials  as  one  of  the  requirements  for  promotion 
within  the  teacher  training  system. 


Implementation  of  the  PIERIM  Model 

Several  specific  areas  of  research  are  suggested  which  relate 
to  the  implementation  of  the  PIERIM  model.  Six  major  areas  of  research 
can  be  considered. 

Behavioral  Objectives 

The  implementation  of  the  PIERIM  Model  resulted  in  the  production 
of  a  set  of  behavioral  objectives  for  the  evaluation  unit  of  an  elemen¬ 
tary  education  course.  Research  is  needed  to  determine  whether  these 
behavioral  objectives  are  the  same  as  a  set  of  behavioral  objectives 
which  would  be  derived  through  a  Job  analysis  based  on  the  tasks  actually 
performed  by  elementary  school  teachers.  If  discrepancies  exist 
between  the  two  sets  of  behavioral  objectives,  are  there  methods  for 
predicting  the  discrepancies  for  other  content  areas? 

Standards  of  Performance 

Rl^search  is  needed  to  determine  the  nature  of  the  relationship 
between  the  standard  of  perfonrince  attained  by  a  learner  and  the 
learner's  performance  on  both  immediate  and  delayed  retention  tests. 
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RBvIilori  of  Instructional  ModuUt 

There  Is  a  need  for  the  development  of  a  simplified  method  of 
ranking  Instructional  modules  as  to  their  relative  need  for  revision 
and  a  rationale  for  terminating  the  revision  process  for  an  Individual 
Instructional  module.  The  preliminary  work  related  to  the  Revision 
Indicator  could  possibly  be  expanded  to  Include  subjective  ratings  by 
the  Instructor  and/or  the  learners.  Research  related  to  the  use  of 
minimum  change  values  In  the  calculation  of  the  Revision  Indicator 
rather  than  the  simpler  dlchotoiiy  which  classifies  observed  changes  as 
being  either  In  a  specified  direction  or  In  the  opposite  direction 
could  possible  Improve  the  sensitivity  of  the  Revision  Indicator. 

A  rationale  Is  needed  for  selecting  the  criteria  to  be  used  to 
terminate  the  revision  process  for  an  Instructional  asdule.  Should  the 
criteria  be  the  same  fbr  Instructional  modules  produced  by  a  selection 
model  and  a  design  model?  The  criteria  of  available  time  and  financial 
resources  between  successive  Implementations  of  the  Instructional  modules 
mutt  be  considered  when  the  design  goals  of  an  Instructional  system  are 
established’. 


Basic  Unit  of  Instruction 


The  basic  unit  used  by  the  PICRIH  model  for  development  and 
revision  was  the  Instructional  module.  The  basic  unit  used  for  evaluation 


of  learner  performance  was  the  set  of  16  Instructional  modules.  It  Is 
necessary  and/or  dHlrable  to  use  the  tame  unit  of  Instruction  for 
development!  Implementation!  evaluation  and  revision.  Research  Is 
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needed  which  would  investigate  the  relative  efficiency  of  using  different 
basic  units  of  instruction  (i.e.,  training  program»  course,  chapter,  task, 
instructional  module,  etc.)  as  the  basis  for  the  development  and  revision 
of  instructional  materials. 

Implementation  Strategy 

Does  the  strategy  used  to  implement  the  instructional  modules 
affect  the  learners'  performance?  If  the  same  set  of  instructional 
modules  were  presented  to  three  group:»  of  learners: 

Group  1  -  the  learners  would  be  given  a  pretest,  interact  with 
the  instructional  modules  and  resources,  and  be  given  a  posttest. 

Group  2  -  the  learners  would  be  given  a  pretest*  exempted  from 
instructional  modules  on  which  the  standard  of  perfonaance  was  achieved. 
Interact  with  the  instructional  modules  and  resources,  and  be  given  a 
posttest. 

Group  3  ~  the  learners  would  be  given  the  instructional  modules 
and  required  to  achieve  the  standard  of  performance  for  each  Instructional 
module,  allowed  to  interact  with  all  of  the  instructional  modules  and 
resources,  and  be  given  a  posttest. 

How  would  the  procedures  used  to  implement  the  instructional  modules 
affect  the  learner's  performance  on  the  immediate  or  a  delayed  retention 
test? 
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Trait  -  Treatment  Interaction 

Approximately  70X  of  the  learners  In  both  groups  indicated  that 
they  would  select  a  course  taught  with  Instructional  modules  rather 
than  the  same  course  taught  without  Instructional  modules.  Can  a  common 
characteristic  or  set  of  characteristics  be  identified  for  the  30%  of 
the  learners  who  preferred  a  course  without  instructional  modules? 

Assume  that  a  common  set  of  characteristics  were  identified  for 
the  learners  who  preferred  traditional  Instruction  to  the  use  of  instruc¬ 
tional  modules  and  self-instruction.  If  the  set  of  characteristics 
did  In  fact  Identify  those  learners  who  would  respond  In  a  similar  manner 
then  the  challenge  to  the  Instructor  and  educational  technologist  would 
be  to  determine  methods  of  acconmodatlng  the  Individual  differences 
within  the  self-instruction  environment.  Redundancy  of  Instructional 
resources  Is  one  possible  alternative  but  It  Is  questionable  if  this  method 
Is  economically  defensible.  An  alternative  area  of  research  would  be 
the  orleitatlon  of  learners  to  the  needs  for  and  purposes  of  Instructional 
research  and  the  validation  of  Instructional  materials.  Would  It  be 
possible  to  design  an  orientation  which  allows  each  learner  to  ask  any 
questions  which  he  felt  were  rel event  and  of  concern  to  him?  What  pro¬ 
cedures  can  be  built  Into  a  self-instruction  environment  which  will 
facilitate  communication  between  the  Instructor  and  the  learner? 
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Instructional  Support  System 

The  research  needed  in  relationship  to  an  ISS  can  be  classified 
into  two  areas: 

1.  production  of  the  instructional  modules  and  tests,  and 

2.  distribution  systems  for  ISS  type  computer  programs. 

Production  of  Instructional  Modules 
and  Tests 

The  cost  of  producing  30  sets  of  the  instructional  modules  and 
tests  for  the  evaluation  unit  was  determined  to  be  approximately  $2.03 
per  set  of  25  instructional  modules  and  tests.  The  major  portion  (i.e., 
91%)  of  the  cost  is  related  to  the  use  of  the  computer  as  the  production 
facility.  Alternative  combinations  of  existing  technology  need  to  be 
considered  in  order  that  the  cost  of  producing  the  instructional  modules 
and  tests  can  be  reduced.  Possibly  the  computer  should  only  be  used  to 
produce  a  master  copy  of  the  instructional  modules  and  tests  and  then 
alternative  methods  of  producing  multiple  copies  of  the  materials  should 
be  considered.  More  efficient  FORTRAN  IV  programming  techniques  could 
possibly  improve  the  efficiency  of  the  ISS  computer  programs. 

Distribution  of  ISS  Type 
Computer  Programs 

The  basic  unit  of  the  ISS  computer  programs  is  the  FORTRAN  IV 

< 

subroutine,  Research  is  needed  to  determine  which  methods  are  currently 
feasible,  the  tradeoffs  between  alternative  methods, and  the  costs 
involved  in  implementing  each  of  the  methods.  One  possible  method  of 
exchanging  subroutine  programs  would  be  the  establishment  of  a  common 


182 


time  sharing  system  for  all  teacher  training  institutions.  The  time 
sharing  system  would  serve  as  a  library  of  instructional  modules  and 
test  items,  each  written  as  a  FORTRAN  IV  subroutine,  and  then  each 
participating  teacher  training  institution  would  have  access  to  the 
most  recent  version  of  each  program  as  it  was  produced. 

Each  of  the  areas  of  needed  research  that  have  been  discussed 
are  relevant  to  future  implementations  and  revision  of  the  PIERIM  Model 
(End  Edition).  The  PIERIM  model  was  designed  specifically  for  use  by 
educators  involved  in  the  preservice  teacher  training  programs.  All 
educators  are  encouraged  to  consider  system  models  for  design  of  instruc¬ 
tion  as  the  underlying  model  for  teacher  training.  When  teacher  training 
programs  have  been  designed  through  the  application  of  system  models 
for  design  of  instruction,  then  future  teachers  can  be  told  to  teach 
as  they  were  taught. 
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